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September 1995 



The following articles represent Test Center holdings to date in the area of assessment alternatives 
in mathematics. Presence on the list does not necessarily imply endorsement; articles are included 
to stimulate thinking and prov ide ideas Some of the entries are informal assessments, and are 
intended mainly for the classroom For more information, contact Matthew Whitaker, Test 
Center Clerk, at (503) 275-9582, Northwest Regional Educational Laboratory, 101 SW Mam, 
Suite 500, Portland, Oregon 97204. e-mail. testcenter@nwrel org. To purchase a copy of this 
bibliography, please call NWREL's Document Reproduction Service at (503) 275-9519. 

.Alberta Education. Diploma Examinations Program — Mathematics 30, 1991. Available 
from: Learning Resources Distributing Centre, 12360 - 142 St., Edmonton, 

AB T5L 4X9, Canada, (403) 427-2767, fax (403) 422-9750. 

•Alberta Education develops diploma examinations in several course areas. These, combined 
with school-awarded “marks” are used to assign credit for the courses. The Mathematics 30 
examination has three parts; multiple-choice, “numerical response" (students “bubble” their 
answers onto the scan sheet), and written response. The test appears to only assess 
knowledge of the subject area rather than problem solving, communication, reasoning, etc 
Examinations are given locally under controlled conditions. Papers are scored centrally 
Scoring appears to be based on the correctness of tlie answer. 

The mathematics test covers advanced algebra, geometry, trigonometry, and number theory. 
(TC# 500.3DIPEXP) 





Alberta Education. 1990. Available from: The Learning 

(403^ '*•’ TSL 4X9, Canada * 

The Dia^wstic Mathematics Program provides assessment and instructional materials for 
hve goal yeas (numyation, operations/properties, measurement, geometry, and problem 
ol mg) at SIX grade levels (1-6) Each handbook contains “obsea-ation checklists " 
structured mten/iews, and written tests for subskills within each skill area "Observations- 
requne teachers to make a judgment of the skill level (strong, adequate, or weak) of the 

assists to identify those students for whom further evaluation is necessary Some 

assistance is given on the criteria for making these judgments ^ 

tflrnTir' ' for a few problems in 

the problem-solving handbook, problems are atomistic— each problem is designed^ to assess a 

atfmi« f “ . fo foe problem-solving handbook, tfsks are 

atomistic (each task is designed to assess a different small skill). Responses are short answer 

and are y ored nght/wong. Total correct is used to assign a rating of “strong,” “adequate,” 

the iTnhi rationale is provided for the cut scores given for each rating In 

s^al^of 0 '?'rn°thJef are scored judgmentally on a hoLtic 

scale of 0-. in three areas: understanding the problem,” "developing and carrying out the 

I^sed • 7^ ir^k"® 7"^^ technical information is provided. These dociSlenIs are best 
used informally by classroom teachers to track progress on individual math skills. 

(TCf^ 500.3DiA,MAP) 



^^fional Assessment of 
of Educ ational Meas..«.m^nt 29 , 

of Educational Progress 

how stu^nf«! a 7' and design of assessments for 1992 and beymd 

^777 7 t results are reported. Although some articles are 

somewhat techmcal, the general pieces on NAEP's history, and the design of current 
assessments will be interesting to the general readership 

Current plans for math include: 

1 Use of calculators for about 70 percent of the test 

2 Estimation skills tasks using an audiotape. 

.1 Yes/no questions to determine the extent to which students understand the 
information when it is presented in different forms. 

4 Constructed response questions in which students are asked to document their solutions 
y drawing their answers, wnting explanations, or providing their computations. 
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Scoring guides for open-ended questions are task-specific Some examples are provided 
(TC# 150.6JEM292) 



Aiper, Lynne. Dan Fendel, Sherry Fraser, et al. Various documents on the Interactive 
Mathematics Program (IMP), 1993-94. Available from: EQUALS, Lawrence Hall of 
Science, University of California, Berkeley, CA 94720, (510) 642-1910, 
fax (510) 643-5757. Information also available from: Linda Witnov, IMP Outreach 
Coordinator. 6400 Hollis St., Suite 5. Emeryville, CA 94608, (510) 658-6400. 

This document includes several papers about the Interactive Mathematics Program which 

»• Discuss the philosophy guiding the program 

• Provide examples of instructional activities, including "Problems of the Week" and student 
self-reflection 

• Report evaluations of the impact of the project on students, impact was measured by the 
SAT and two attitude measures (which are included); (no performance assessment was 
used as an outcome measure, the performance assessments described in the materials 
include tasks but no performance criteria) 

Some sample student work is included 

(TC# 500.3VARIMP) 



Appalachia Educational Laboratory. Alternative Assessments in Math and Science: Moving 
Toward a Moving Target, 1992. Available from: Appalachia Educational Laboratory, 
PO Box 1348, Charleston, WV 25325, (304) 347-0400. 

This document reports a two-year study by the Virginia Education Association and the 
Appalachia Educational Laboratory which twenty-two K- 1 2 science and math teacliers 
designed and implemented new methods of evaluating student competence and application of 
knowledge. Teachers who participated in the study felt that the changes in assessment 
methods led to changes in their teaching methods, improvements in student learning and better 
student attitudes. Instruction became more integrated across subjects, instruction shifted from 
being teacher-driven to being student-driven, and teachers acted more as facilitators of 
learning rather than dispensers of information. 

Included in the report is a list of recommendations for implementing alternative assessments, a 
list of criteria for effective assessment, and 22 sample activities (with objectives, tasks, and 
scoring guidelines) for elementary, middle, and high school students, all designed and tested 
by the teachers in the study 
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Most activities have performance criteria that are holistic and task-specific No technical 
information or sample student work is included 

(TC# 600.3 ALTASM) 



Arizona Student Assessment Program. Mathematics: Expressions and Equations: 
Mathematics: Identifying Shapes; Mathematics: Statistics; Writing: Specialized 
Expository Paper; Writing: Persuasive Paper; and Reading: Poem; 1992-93. Available 
from: The Riverside Publishing Company, 8420 Bryn Mawr Ave., Chicago, IL 60631, 
(800) 323-9540. 

Arizona is developing open-response, on-demand assessments in math, reading, writing, social 
studies, and science tor grades 3, 8. and 12 The math test has been used for about two years, 
the science test is still under development The mathematics test requires a minimum of 
manipulatives Students provide short answers to a series of questions surrounding a common 
theme, such as a rock climbing competition or a "pizza spinner " They also sometimes 
provide explanations for responses or show computations. 

Scoring is based on a four-point general holistic rubric The correct answers are given for 
individual parts of the activity and the reader assigns 0-4 points depending on understanding 
the problem, logical reasoning, effective communication, and correctness and completeness of 
the response 

The tests were developed by Riverside and are under copyright restriction until at least 1996 
Permission to use any part of the assessment must be granted by both Riverside and the 
.Arizona Department of Education 

(TC# 500.3ARISTM2) 



.Aurora Public Schools. Performance Assessments in Science and Mathematics, 1992. 
Available from: Strategic Plan Facilitator, Aurora Public Schools, Department of 
Instructional Services, 15751 E. 1st Ave., Suite 220, Aurora, CO 80011, (303) 340-0861, 
fax: (303) 340-0865. 

The author has provided three high school examples of the types of assessments being 
developed by teachers in Aurora Public Schools: developing an analogy for the major 
anatomical and physiological components of a typical eukaryotic cell, recommending a 
decision concerning the future use of a medical technology in human biology, and collecting 
and analyzing a data set These examples include a description of the task, prerequisite 
student experiences, and criteria forjudging student performance on the task Students work 
in groups of two to four The assessments are mostly for classroom use. 

Performances are evaluated along several dimensions including content, complex thinking, 
decision making, and collabora i\e working. Most of the rubrics are task specific and 
emphasize relative quality For example, a “4” score for complex thinking on the medical 
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technology task is: “The student clearly and completely identified the criteria by which the 
alternatives were assessed. The criteria were presented in detail and reflected an unusually 
thorough understanding and concern for the repercussions of the decision ” The collaborative 
worker rubric is generic and more descriptive, a "4" is "The student expressed ideas clearly 
and effectively, listened actively to the ideas of others, made a consistent effort to ensure that 
ideas were clearly and commsnly understood, accurately analyzed verbal and non-verbal 
communications, solicited and showed respect for the opinions of others ' 

No technical information nor sample student responses are included 

(TC# 000.3SCIMAP) 



Badger, Elizabeth. On Their Own: Student Response to Open-Ended Tests in Mathematics, 
1989-1991. Available from: Massachusetts Educational Assessment Program, 
Massachusetts Department of Education, 350 Main St., Malden, MA 02i48, (617) 388- 
3300. 

This document contains assessment materials for grades 4, 8, and 12 from three years (1988- 
1990) in four subject areas (reading, social studies, science and math). In 1988 and 1990 
students solved problems and explained their answers These problems emphasized the major 
areas of patterns/relationships, geometry/measurement, and numerical/statistical corxepts. All 
problems were written. Responses were scored both for correctness of solution and for 
quality of the explanation No specific criteria for judging quality of explanation were given. 
Several examples of student responses illustrating various conclusions are included. 

In 1989, 2,000 students were assigned one of seven performance tasks (four in math required 
manipulatives) to do in pairs Each pair was individually watched by an evaluator. It took 65 
evaluators five days to observe the 2,000 performances Evaluators checked off those things 
that students did correctly (e g , measure temperature), and recorded students' conversations 
and strategies Specific criteria are not included. 

Some information on results for all the assessments is provided; percentages of students 
getting correct answers, using various strategies, using efficient methods, giving good 
explanations, etc., depending on the task. Many examples of student responses illustrating 
these various points are provided. No technical information about the assessments themselves 
is provided. 

(TC# 500.3ONTHOM) 



Bagley, Theresa, and Catarina Gallenberger. Assessing Students' Dispositions: Using 
Journals to Improve Students' Performance. Located in: The Mathematics Teacher 85, 
November 1992, pp. 660-663. 

The authors discuss the use of journals to elicit student behavior that can be examined for high 
school student attitude toward math, at ility to make mathematical connections, and 
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understanding. They present many questions, tasks, and instructions for getting students to 
self-reflect, and provide good, practical suggestions for managing the process. However, the 
authors do not provide criteria for examining student responses — what to look for in 
responses that are indicators of attitude, connections or understanding Therefore, the papers 
will beusefijl only to the extent that users have the expertise to know what to look for in 
responses 

(TC# 500.6ASSSTD) 



Baxter Gail P., Richard J. Shavelson, Sally J. Herman, Katharine A. Brown, and James R. 
V aladez. Mathematics Performance Assessment; Technical Quality and Diverse Student 
Impact Located in; Journal for Research in Mathematics Education 24. 1993. 
pp. 190-216. 

The authors developed 41 hands-on tasks to measure three categories of sixth-grade student 
competencies: measurement (seven tasks), place value (3 1 tasks), and probability (three 
tasksV An example of a measurement task is "describe the object." in which students write a 
description of an object that someone else could use to draw the object. An example of a 
probability task was "spin it" in which students were given a spinner with eight sections (four 
orange, three yellow, and one green) They had to predict which color the pointer would land 
on most or least often, predict the outcome of 32 spins, carry out the experiment, and graph 
the results 



Responses were scored either by degree of "correctness" or, in the case of the communication 
items (e g., describe an object), holistically for general quality of the response The tasks and 
criteria are described only in general terms, fUrther information would have to be obtained 
from the authors in order to actually reproduce the assessment. 

Tasks were pilot tested with 40 sixth graders (Anglo and Hispanic) two types of instructional 
settings hands-on and traditional. Results showed: raters using this type of rating scheme 
can be trained to be very consistent in their scoring; the assessments are costly and time- 
consuming, a considerable number of tasks need to be administered to provide a reliable 
estimate of a student's level of achievement; student performances on the hands-on tasks 
differed by the type of instructional setting (evidence of validity), and there was differential 
scoring on the part of Hispanics, leading to some equity concerns 

(TC# 500.6MATPEA) 



Bellingham Public Schools. Primary Performance Portfolio, Grades K, I, 2 and Intermediate 
Performance Portfolio, Grades 3, 4, 5, 1992. Available from: Bellingham Public 
Schools, Box 878, Bellingham, WA, 98227, (360) 676-6400, fax (360) 676-2793. 

This document is an outline of suggestions for implementing portfolios for grades 1-5 
Included are 
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• A list of essential learnings in each grade level combined with suggestions of the type of 
information that could be included as evidence of progress on each goal For example, a 
student goal in grades 3-5 is "literature competency ” Portfolios could include response 
logs, performance or project scoring rubrics, or strategy checklists as evidence of 
attainment of this goal 

• Checklists and scoring guides for some of the skills For example, fairly nice development 
continuums for reading, science, social studies, art. writing and mathematics are included, 
each having six levels (pre, emergent, beginner, developing, capable, and experienced) 
These are incorporated into a report card 

.Also included are some materials for two units history and energy It is not clear how this 
material is used nor why it is included No technical information nor samples of student work 
are included 

(TC# 000.3BELPI S) 



British Columbia Ministry of Education. Performance Assessment: Primary, Early, Late, 
Intermediate, and Graduate, and HyperCard Tour. Draft, August 1992. Available from: 
Ministry of Education, Assessment Branch, 617 Government St., Victoria, British 
Columbia V8V 2M4. 

This is a Macintosh disk containing a host of performance assessments developed by the 
British Columbia Ministry of Education for all grade levels and subject matter areas 

(TC# 000.3BCPERA) 



Burns, Marilyn. Math and Literature (K-3), 1992. Available from: Math Solutions 

Publications, Marilyn Burns Education Associates, 150 Gate 5 Rd., Suite 101, Sausalito, 
CA 94965, (415) 332-4181, fax (415) 331-1931. 

The author's premise is that "children's books are effective classroom vehicles for motivating 
students to think and reason mathematically." The book describes instructional ideas for 
linking math and literature in grades K-3 Samples of student work are included. The book is 
included on this bibliography because it might provide ideas for tasks that would be useful in 
performance assessments 

(TC# 500.5MATLIT) 
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Burton, Grace, Terrence Coburn, John Del Grande, et al. Addenda Series, Grades K-6: 
Curriculum and Evaluation Standards for School Mathematics, 1991-92. Available from: 
National Council of Teachers of Mathematics, 1906 Association Dr, Reston, VA 22090, 
(800) 235-7566. 

Addenda is a series of booklets to suppon the NCTM Standards This series provides 
instructional ideas tor grades K-6 tor the areas of patterns, geometry/spatial sense, rriaking 
sense of data, and number sense/operations The reference is included here because the 
exercises might suggest tasks that could be used in performance assessments 

(TC# 500.5ADDSEK) 



California Assessment Collaborative. Charting the Course Toward Instructionally Sound 
Assessment A Report of the Alternative Assessment I^lot Project, September 1993. 
Available from: California Assessment Collaborative, 730 Harrison St., San Francisco, 
CA 94107,(415) 241-2704. 

The California Assessment Collaborative was designed as a three-year effort to systematically 
identify’, validate, and disseminate alternatives to standardized testing throughout the state 
This report presents findings from 22 projects during the first year, including costs, impacts, 
and recommendations about future work. The book does an excellent job of placing 
assessment change into the context of restnacturing It discusses how the following fit 
to ether articulating content standards, monitoring student progress toward goals, building 
teacher capacity to assess, building student capacity to self-assess, student outcomes, 
curriculum, and instruction 

(TC# 150.6CHACOI) 



California Assessment Program. A Sampler of Mathematics Assessment - Addendum - 
Preliminary Edition, 1993. Available from: California Department of Education, PO 
Box 944272, Sacramento, CA 94244. 

The Sampler Addendum supplements a previous publication (see Pandey on this 
bibliography). While the previous publication provided an overview of the new assessment 
system, the addendum provides scoring guides and scored samples of student work in grades 
4, 8 and 10. 

(TC# 500.3SAMMAA2) 
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California Department of Education. Students, Standards and Success — The California 
Learning Assessment System — Communications Assistance Packet, 1993. Available 
from: California Department of Education, Publications Sales, PO Box 271, 

Sacramento, CA 95812, (916) 657-3747 or (800) 995-4099. 

The purpose of this document is to introduce the California Learning Assessment System 
(CLAS) to the public It gives an overview of the system and helps with setting up 
communication efforts. CLAS is administered in i!ie spring at grades 4, 8, and 10 in reading, 
writing, and mathematics, and grades 5, 8, and 10 for science and social studies There is also 
a portfolio component CLAS is designed to give information on individual students, is tied 
to state standards, and is performance based State law also specifies that CLAS links to 
curriculum frameworks and guides and should exemplify good instruction, including student 
engagement, thinking, problem solving, and communication The document includes letters to 
parents, answers to frequently asked questions, sample press releases, sample meeting 
agendas, information on the assessments, resources, and sample items and student responses 

The math scoring guide is a four-point, holistic rubric emphasizing conceptual understanding 
and clear communication The reading/writing scoring guide is a six-point, holistic scale that 
focuses on constructing meaning, developing connections, taking risks, and challenging the 
text The writing scoring guide is a six-point, holistic scale that emphasizes position, 
organization, coherence, elaboration, relevance of arguments, awareness of style, and sentence 
variety 

(Note: California is currently reviewing its large-scale assessment, CLAS may not continue ) 
tTC# 000.6STUSTS) 



California State Department of Education. A Question of Thinking: A First Look at 

Students' Performance on Open-Ended Questions in Mathematics, 1989. Available from: 
California State Department of Education, PO Box 944272, Sacramento, CA 94244- 
2720, (916) 445-1260. 

This report describes the results of a 12th grade student assessment using open-ended math 
problems. Scoring was task-specific and student work is included This procedure was 
replaced by the later California entries on this bib It is still included because others might find 
the problems useful. 

(TC# 500.3AQUESO) 



Carpenter- Thomas P., James Hiebert, Elizabeth Fennema, ct al. A Framework for the 
Analysis of Teaching and Learning Understanding of Multidigit Numbers. Information 
on date ^nd availability is unknown. 

This paper presents a way to analyze instruction in math to see whether it is designed to foster 
understanding, defined as making relevant connections between knowledge The specific 
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example in the paper relates to multidigit numbers Dimensions of instruction thought to be 
critical in promoting understanding include the scope and sequence of concepts, connections 
among representations as a basis for establishing meaning for symbols, the nature of problem 
solving, degree of teacher specification of solution procedures and connections, students' 
articulation of solution procedures, and coherence between and within lessons 

Most of the paper describes each of these dimensions in detail Several pages at the end 
discuss in general terms the kinds of tasks one could give to students to see whether they are 
making the appropriate connections 

(TC# 500.4FRAANT) 



Center for Talent Development. Elementary School Pre-Post Survey and Middle/High 
School Pre-Post Survey, 1991 .Available from: Evaluation Coordinator, Center for 
Talent Development, Northwestern University, Andersen Hall, 2003 Sheridan Rd., 
Evanston, IL 60208, (708) 491-4979. 



This document contains surveys of student attitudes toward mathematics and science. There 
are two levels elementary and middle/high school It was designed for use with Access 2000 
participants who are primarily African-American and Hispanic students in an inner-city public 
school ysiem and enrolled in a math/science/engineering enrichment program No technical 
information is included 

(TC# 22v.30’JFFLM) 



Champagne Audrey B. Cognitive Research on Thinking in Academic Science and 

Mathematics: Implications for Practice and Policy. Located in: Enhancing Thinking 
Skills injiLe Sc iences and Mathematics . Diane Halpern (Ed.), 1992. Available from: 
Lawrence Eiihaum Associates, Publisher, 365 Broadway, Hillsdale, NJ 07642, 

(800) 926-6579. 

/\ltho"gh this article is not strictly about assessment, it discusses some topics of relevance to 
assessment. Specifically, it has a very nice section on the relationship between the tasks given 
to students and what they learn For example, students have more trouble integrating 
knowledge if they are never given tasks that require them to do this. Likewise, one can't 

assess a skill such as integrating knowledge if the assessment task doesn't require students to 
do it 

(TC# 000.6COGRET) 
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Charles, Randall, Frank Lester, and Phares O'DafTer. How to Evaluate Progress in Problem 
Solvings 1987. Available from: National Council of Teachers of Mathematics, 1906 
Association Dr., Reston, VA 22091. 

This monograph attempts to assist educators with the challenge of developing new techniques 
for evaluating the effectiveness of instruction in problem solving by clarifying the goals of 
problem-solving instruction, and illustrating how various evaluation techniques can be used in 
practice. Goals include: select and use problem-solving strategies, develop helpful attitudes 
and beliefs, use related knowledge, monitor and evaluate thinking while solving problems, 
solve problems in cooperative learning situations, and find correct answers. 

Evaluation strategies include informal observation/questioning and recording results using 
anecdotal records or a checklist (two are provided); interviews (a sample inter/iew plan is 
provided), student written or oral self-report of what's happening during a problem-solving 
experience (a list of stimulus questions is given, as is a checklist of strategies); attitude 
inventories (two are given); rating scales (three-trait analytic and focused holistic scales are 
given), and multiple-choice and completion (sample items are given to assess various problem- 
solving abilities) No technical information nor student sample performances are provided. 

(TC# 500.6HOWTOE) 



Clark, David. The Mathematics Curriculum and Teaching Program, 1988. Available from: 
Curriculum Development Centre, PO Box 34, Woden, ACT 2606, Australia. Also 
available from: ERIC ED 287 722. 

This document was developed to assist classroom teachers in grades 1-6 to improve their day- 
to-day assessment of mathematics Content includes: rationale for assessment ".Itematives in 
mathematics, instructions for a two-day inservice program using the materials, instructions on 
how classroom teachers can use the materials without training, and a series of exercises, 
formats and ideas for classroom assessment. Assessment ideas include: help with 
systematically recording information from informal observations using checklists and "folios" 
of student work, setting-up opportunities for assessment by giving students good tasks to do, 
assessing problem solving, student self-reflection, and communicating results. • 

This is written in a very user-friendly manner and contains some good ideas, especially in the 
areas of designing tasks, problem solving and self-reflection. We found some of the 
descriptions of activities a little too sketchy 

(TC# 500.3MCTPMA) 



Clark, Glenda. AIM High Math Identification, 1991 Austin Independent School District 
Gifted and Talented Program, '’HE. 7th St., Austin, TX 78701, (512) 476-6861. 

This set of paper-and-pencil tasks was developed as one of five indicators for placing grade 2- 
5 students in the Austin Independent School District gifred math program Each task has 
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multiple open-response questions related to a common theme. Responses are scored on a 
scale of 1-5 using task-specific scoring guides. The document includes one task per grade 
level, the scoring guides, and student work samples. Not included are technical information 
or contextual information describing what the assessments are trying to measure 

(TC# 500.3AIMHIM) 



Clarridge, Pamela Brown and Elizabeth M. Whitaker. Implementing a New Elementary 
Progress Report. Located in: Educational Leadershin . October 1994. nn. 7-9. Also 
available from: Tucson Unifled School District #1, 1010 E. Tenth St., 

Tucson, AZ 85719. 

This paper reports on one district's attempt to revise its report card for grades K-5 Staff 
decided on a rubric approach. In grades 1-5, rubrics using four-point scales were developed 
for five "learner qualities" — self-directed learner, collaborative worker, problem solver, 
responsible citizen, and quality oroducer, and eight content areas — reading, writing, 
listening/speaking, mathematics, social studies, science, health, and fine arts. Room is 
provided on the report card for teacher comments, the basis for the judgment about student 
ability (e g , classroom observation, portfolios), and teacher/student comments 

The authors describe development and pilot testing, preliminary responses from parents and 
students, plans for revision, and insights (such as "this approach to reporting requires a 
thorough understanding of the curriculum by both parents and teachers") 

(TC# 150.6IMPNEE) 



Collis, Kevin F. and Thonas Romberg. Collis- Romberg Mathematical Problem Solving 
Profiles, 1992. Available from: Australian Council for Educational Research Limited, 
(ACER), Private Bag 55, Camberwell Victoria 3124, Australia, (03) 277-5555, 
fax: (03) 277-5500. Also available from: ASHE, PO Box 31576, Richmond, VA 23294, 
(804) 741-8991. 

This assessment device for students in grades 2-5 has 20 open-ended problems to solve — one 
problem in each of five areas (algebra, chance, measurement, number, and space) with four 
questions per problem area designed to tap developmental level of formal reasoning. For 
example, the "A" question determines whether the student can use one obvious piece of 
information from the item, while the "D" question determines whether the student can use an 
abstract general principle or hypothesis derived from the information in the problem. 
Responses to each question are scored right/wrong. The number of correct responses 
determines a developmental level Suggestions are given for instructional strategies for the 
various developmental levels. Technical information in the manual includes typical 
performance for various grade levels, teacher judgment on the developmental level indicated 
by each task, and additional analyses to show validity of the inferences drawn. 

(TC# 500.3COLROM) 
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Cominission on St&nd&rds for School Mathematics. Cutnculuni ond Evoluotion Stondofds 
for School Mathematics, 1989. Available from: National Council of Teachers of 
Mathematics, 1906 Association Dr., Reston, VA 22091. 

This book contains standards for curriculum and assessment that attempt to create a coherent 
v ision of what it means to be mathematically literate. This book has been quoted extensively 
and is the current "standard" for what should be in a math curriculum. 

The assessment section covers three statements of philosophy concerning assessment 
(alignment, multiple sources of information, and appropriate assessment methods and uses); 
seven sections on assessing various student outcomes (e g., problem solving, communication, 
reasoning, concepts, procedures, and dispositions); and four sections on program evaluation 
(indicators, resources, instruction, and evaluation team) Each of the seven sections on 
assessing student outcomes briefly describes what the assessment should cover and provides 
some sample assessment tasks and procedures. 

(TC# SOO.SCURANE) 

Conley, David T. Proficiency- Based Admission Standards, January 8, 1995. Available 
from: PASS Project, Oregon State System of Higher Education, OfTice of Academic 
.Affairs, PO Box 3175, Eugene, OR 97403, (503) 346-5799. 

This paper describes the Oregon Board of Higher Education's new policy on admitting 
students by demonstration of competencies rather than number of courses taken or GP A. 
Included is the rationale for the approach (including the incongruity between traditional 
college admissions procedures and the attempt by K-12 schools to restructure), a list of the 
competencies, ideas for assessment, ideas for how high schools might need to change in order 
to ensure students meet admissions standards, and commonly asked questions. Competencies 
include subject area content (science, math, etc.) as well as basic and process skills (writing, 
reading, critical thinking, etc ). 

The paper addresses the concern by some parents and teachers that changes in K-12 education 
won't mean anything if students are still admitted to college using traditional methods. The 
authors point out that similar changes in college admissions policy are occurring nationwide. 

(TC# 150.6PROBAA) 

Connecticut State Department of Education. Connecticut Common Core of Learning 
Assessment, 1989-1992. Available from: Connecticut State Department of Education, 
Division of Research, Evaluation, and Assessment, 165 Capitol Ave., Room 340, 
Hartford, CT 06106, (203) 566-4001. 

This package contains a variety of documents produced between 1989 and 1992. Included 
are information about; rationale for the assessment, Connecticut's Common Core of Learning 
(student learning objectives), development process, several sample tasks and scoring 
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mechanisms, student and teacher feedback forms, summaries of student and teacher feedback 
on the assessments using these forms, a process for developing performance tasks, a survey 
for student attitudes about science and mathematics, and an example of concept mapping as 
an assessment tool 

There appear to be two kinds of tasks: complex group projects and shorter on-demand tasks 
covering individual skills The projects attempt to get at application and extension of 
knowledge and concepts They require some individual and some group work and extend at 
least over several days. The on-demand portion covers knowledge and it's application in 
limited situations Performances are scored for group collaboration, process skills, and 
communication skills Some of the rubrics are task specific and some are general; some are 
based on quantity (the number of possible solutions listed, for example) and some are more 
quality based 

(TC# 000.3CONCOC) 



Crowley, Mary L. Student Mathematics Portfolio; More Than a Display Case. Located in: 
The Math ematics Teacher 86, October 1993, pp. 544-547. 



The author discusses the use of mathematics portfolios to document progress on the "big" 
NCTM outcomes— problem solving, valuing mathematics, developing mathematical 
confidence, communicating mathematically, and reasoning mathematically. She includes a fair 
amount of detail on a middle school example — sample letter to students outlining the task, a 
description of what students need to include and a hint of how content might be assessed ’ 

The portfolio is semi-structured — the teacher specifies categories of entries and the student 
selects which samples of work to include in each category. Students must also write 
explanations for their selections. A few samples of student work are included. Detailed 
performance criteria are not included. 

(TC# 500.3STUMAP) 



Csongor, Julianna E. Mirror, Mirror On The Wall.. Teaching Self Assessment to Students. 
Located in: The Mathem atics Teacher 85, November 1992, pp. 636-637. Also available 
from: Saint Maria Gosetti High School, 10th and Moore, Philadelphia, PA 19148. 

The author presents a procedure for getting high school students to self-reflect in math: 
during the final five minutes of a test, students estimate how sure they are about each answer 
they gave on the test (100%, 75%, 50%, or 0%). They can earn extra credit on the test if 
their estimates fall within 3% of their actual score. She reports that students are surprisingly 
accurate in their estimates and that the procedure works especially well with slow learners 

(TC# 500.3MHIMIW) 
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CTB McGraw-Hill. CAT/5 Performance Assessment Component, 1993. Available from: 
CTB/McGraw-Hill, 2500 Garden Rd.. Monterey, CA 93942, (LOO) 538-9547, 
fax (800) 282-0266. 

The "CTB Performance Assessments" are designed to either be stand-alone or integrated with 
the CAT'5 or CTBS/4 There are five levels for grades 2-1 1 The total battery includes 
reading/language arts, mathematics, science, and social studies and takes 2-3 hours to 
administer There are 12-25 short- to medium-response questions for each subtest The math 
and science subtests take 30-40 minutes (For the CAT/5 there is a checklist of skills that can 
be used at grades K and 1 ) 

Some questions are grouped around a common theme Many resemble multiple-choice 
questions with the choices taken off For example, questions on one level include; “What are 
two ways that recycling paper products helps the environment'’” “This table shows the air 
temperatures recorded every two hours from noon to midnight. At what time did the 
temperature shown on the thermometer most likely occur'’” and “These pictures show some 
of the instruments that are used in science .List two physical properties of the water in the jar 
below that can be measured with the instruments shown in the pictures. Next to each 
property, write the name of the instrument or instruments used to measure the property .” 

Some of the answers are scored right/wrong and some are scored holistically The materials 
we received contained no examples of the holistic scoring so we are unable to describe it. 
Scoring can be done either locally or by the publisher When the Performance Assessments 
are given with the CAT/5 or CTBS/4, results can be integrated to provide normative 
information and scores in six areas. There are only three, however that use the math and 
science subtests: demonstrating content and concept knowledge, demonstrating knowledge of 
processes/skills/procedures, and using applications/problem-solving strategies. When the 
Performance Assessments are given by themselves, only skill scores are available 

The materials we received contain sample administration and test booklets only. No technical 
information or scoring guides are mcluded. 

(TC# 060.3CAT-5a) 



CTB McGraw-Hill. CTB Math Task Bank, uninttd. Available from: CTB/McGraw-Hill, 
PO Box 150, Monterey, CA 93942, (800) 538-9547, fax (800) 282-0266. 

The “CTB Math Task Bank” contains 644 math tasks for grades 3-9 (There is also a 
language arts bank.) Each task has a series of questions around a theme. For example, the 
ninth grade bank contains a series of questions that relate to “selling advertisements for the 
yearbook.” Questions require students to do such things as: identify how many ads of 
various sizes can fit on a single page, how much money different types of layouts can 
generate; and how many ads need to be sold to cover expenses Most tasks are, thus, very 
structured and have only one right answer However, a few are more open-ended. For 
example, a grade 4 task is “You and a friend are playing a guessing game You think of the 
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number 10 Your fnend must guess the number you have chosen. Give your friend some 
hints to help him guess this number. On the lines below, write four different hints about this 
number using math ideas.” Additionally, some tasks attempt to get at self-reflection: “Draw a 
circle around the number below that shows how easy or how hard it was for you to solve the 
problems in this task ” 

The materials we received do not elaborate on scoring. It appears that most questions are 
scored right/wrong Presumably then, the total number of points on questions covering each 
skill provide an indication of student ability. No direct Judgments of problem solving, 
reasoning, communication, etc are made 

Tasks can be accessed by student learning goal (tasks are cross-referenced to the NCTM 
standards), theme (e g . year book ads), or question difficulty CTB also publishes software to 
support the task bank 1 his includes test generation and scoring. The materials we received 
did not contain technical information 

(TC# 500.3CTBMAT) 



Curriculum Corporation. Mathematics — A Curriculum Profile for Australian Schools, 
Mathematics — Work Samples, and Using the Mathematics Profile, 1994. Available from: 
Curriculum Corporation, St. Nicholas PI., 141 Rathdowne St., Carlton, Victoria, 3053, 
Australia, (03) 639-0699, fax (03) 639-1616. 

These documents represent the mathematics portion of a series of publications designed to 
reconflgure instruction and assessment in Australian schools. The project, begun in 1989, was 
ajoint effort by the States, Territories, and the Commonwealth of Australia, initiated by the 
Australian Education Council. 

The profiles are not performance assessments, per se. in which students are given 
predeveloped tasks Rather, the emphasis has been on conceptualizing major student 
outcomes in each area and articulating student development toward these goals using a series 
of developmental continuums. These continuums are then used to track progress and are 
overlaid on whatever tasks and work individual teachers give to students. 

The math profiles cover the major strands of working mathematically, space, number, 
measurement, chance/data, and algebra. Each strand has sub-areas c^led “organizers ” For 
example, the strand of “working mathematically” contains the organizers of; investigating, 
conjecturing, using problem-solving strategies, applying and verifying, using mathematical 
language, and working in context Each organizer is tracked through eight levels of 
development For example, the organizer of “using mathematical language” has “talks about 
mathematical ideas in natural language ’ at Level 1 , and “makes fluent use of mathematical 
notation in solving problems and presenting arguments succinctly, coherently and in 
conventional forms” at Level 8. 

There are lots of support materials that describe what each strand means, how to organize 
instruction, types of activities to use with students, and how to use the profiles to track 
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progress Samples of student work are included to illustrate development. The documents 
say that the levels have been ‘validated,’’ but this information is not included in the materials 
we received 

(TC# 500.3MATCUP) 



Doig, Brian. Activities and Assessment in Mathematics (AAIM), 1995. Available from: The 
Australian Council for Educational Research Ltd., 19 Prospect Hill Rd., Camberwell, 
Melbourne, Victoria .>124, Australia, 03-819-1400. 

The AAIM is a collection of over 200 math tasks which cover the five strands of the 
Australian Curriculum Profile — number, space, measurement, chance/data, and problem 
solving It is designed for students in upper elementary and lower secondary (grades 4-9) 

The author suggests two primary uses — diagnosis of student skills in specific content areas, 
and tracking student development along the curriculum strands. All scoring is task specific. 
Since tasks are keyed to the stages of the curriculum profile, getting a high score on a task is, 
presumably, an indicator of student acquisition of skill at the level to which the task relates 
The document includes a user's manual and all 200 tasks and associated scoring guides. No 
sample student work nor technical information is included 

(TC# 500.3ACTASM) 



Educational Testing Service. Miscellaneous alternative assessments, 1993. Available from: 
Educational Testing Service, 1979 Lakeside Pkwy, Suite 400, Tucker, GA 30084, 

(404) 723-7424. 

Six teams of elementary and middle schools in Georgia, in conjunction with the ETS Southern 
Field Office in Atlanta, are working on math and science assessment activities (cooperative 
group, videotape, open-ended experiments) that can be used across grades and content ai eas, 
and that are designed to assess science process skills, math problem solving, ability to 
communication in science and math, and content knowledge 

The materials we have include scoring guides (both general and task-specific, and holistic and 
analytical trait) and scored samples of student work. No technical nor contextual information 
is included in the materials (although such information is available from the publisher) 

Any use requires permission from Educational Testing Service 

(TC# 000.3MISALA) 
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Educational Testing Service. NAEP 1 992 Mathematics Report Card for the Nation and the 
States, April 1993, Report No. 23-ST02. Available from: Education Information 
Branch, Office of Educational Research and Improvement, US Department of 
Education, 555 New Jersey Ave NW, Washington, DC 20208, (800) 424-1616 or 
(202) 219-1651. 

The National Assessment of Educational Progress (NAEP) is congressionally mandated 
Tests are given in reading, writing, math, science, social studies, fine arts, writing, literature, 

career development, and computers to students in grades 4, 8, and 12 on a revolving basis 

not all subject areas are given each year. This entry describes the 1992 math assessment 
which tested approximately 220,000 students from 9,000 schools 

The on-demand tests covered numbers/operations, measurement, geometry, data 
analysis/ statistics/probability, algebra/flinctions, and estimation. These content areas were 
covered at three levels conceptual understanding, procedural knowledge, and problem 
solving. Some questions were multi-step Calculators were allowed. There were both 
multiple-choice and short answer (e g., measure an angle and write in the measurement) 
questions. Some of the questions required short explanations. For example, one fourth grade 
question asked students to choose the graph that represented the number of pockets in the 
clothing worn by a class ot 20 students and provide an explanation of why they picked the 
graph they did Responses were scored right/wrong 

The 1992 math assessment marks a shift to reporting by proficiency levels. For example, the 
Basic” level designates concepiual and procedural understanding, while the “Advanced” level 
represents the ability to generalize and synthesize concepts and principles (Note. The setting 
ot cut-ofT scores on the tests relative to these levels has been controversial from a technical 
standpoint and will undergo further refinement ) 

The report contains NAEP background information on the achievement levels and how they 
were set, sample items illustrating the levels, and lots of tables of results by states, regions, 
and various demographics such as gender and ethnicity Released sets of items are available in 
other documents. 

(TC# 500.6NAEPMAR) 



EQUALS. Assessment Alternatives in Mathematics, 19S9. Available from: University of 
California, Lawrence Hall of Science, Berkeley, CA 94720, (415) 642-1823. 

This document provides an overview of some possible assessment methods in mathematics 
that cover both process and products. Specific examples are given for writing in mathematics, 
mathematical investigations, open-ended questions, performance assessment, observations, 
interviews, and student self-assessment. Any of the student-generated material could be self- 
selected for a portfolio of work The document also includes a discussion of assessment 
issues and a list of probing questions teachers can use during instruction 

(TC# 500.6ASSALI) 
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Ferguson, Shelly. Zeroing in on Math Abilities, 1992. Located in: Learning92 . Vol. 21. 
pp. 38-41. 



This paper was written by a fourth grade teacher and describes her use of portfolios in math — 
what she has students put in their portfolios, the role of self-reflection, getting parents 
involved, and grading She gives a lot of practical help One interesting idea in the paper has 
to do with grading. At the end of the grading period she reviews the portfolios for attainment 
of concepts taught (not amount of work done), and progress toward six goals set by the 
NCTM standards (e g., thinks mathematically, communicates mathematically, and uses tools) 
She marks which goals were illustrated by the various pieces of work in the portfolio and 
writes a narrative to the student Another interesting idea is formal presentations of their 
portfolios by students to their parents The article provides a sample comment form for 
parents and students to complete 

(TC# 500.3ZERMAA) 



Finston, D., A. Knoebel, and D. Kurtz. Student Assessment Using Student Research 
Projects, 1993. Available from: Douglas S. Kurtz, Professor of Mathematics, 
Department of Mathematical Sciences, New Mexico State University, Box 30001, 

Las Cruces, NM 88003, (505) 646-6218. 

This program (used in grades 9-university) is based on "student research projects,” multi-step 
assignments lasting up to several weeks, which focus on problem-solving skills and writing. 
Students work in groups of 3-4 Sample projects in the document include remodeling a 
house, deciphering averages, determining tunnel clearances, and designing revolutionary 
solids. Assessment of student work is based on teacher evaluation of written reports. The 
sample scoring guides included in the document generally include content, process, and 
communication skil ., but are fairly sketchy. No technical information is included. 

The author is in the process of preparing a book compiling information on the student 
research projects, but grants permission to educators for reproduction. 

(TC# 500.3STUASU) 



Fitzpatrick, Robert and Edward J. Morrison. Performance and Product Evaluation. 

Located in: Educational Performance Assessment. Fredrick L. Finch (Ed.), 1991. 

Available from: The Riverside Publishing Company, 8420 Bryn Mawr Ave., Chicago, 

IL 60631, (800) 323-9540. 

This paper has interesting discussions of the following topics; 

1 What "authenticity" in tasks means. The authors' position is that there are many degree ■'> 
and kinds of artificialities in tests. "Performance and product evaluation are those in 
which some criterion situation is simulated to a much greater degree than is represented 
by the usual paper-and-pencil test .. [However,] there is no absolute distinction 
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between performance tests and other classes of tests — the performance test is one that is 
relatively realistic." 

2 Criteria for deciding how much "reality" to include in tasks 

3 Descriptions of v arious types of tasks that can be used in performance assessments 
in-basket, games, role-plays, projects, etc 

4 Steps for dev eloping performance assessments analysis of the important dimensions of 
the skills to be covered, identification of tasks that cover as many of the important skills 
as possible, developing instructions and materials, and developing the scoring procedure 

Most specific examples are taken from military and business applications 

(TC# 150.6PERPRE) 



Flexer, Roberta J. and Eileen A. Gerstner. Dilemmas and Issues for Teachers Developing 
Performance Assessments in Mathematics— A Case Study of the Effects of Alternative 
Assessment in Instruction, Student Learning and Accountability Practices, CSE Technical 
Report 364, October 1993. Available from: National Center for Research on 
Evaluation, Standards, and Student Testing (CRESST), Graduate School of Education, 
University of California, Los Angeles, CA 90024, (310) 206-1532. 

This monograph is the first in a series reporting on an ongoing study of the implications of 
alternative assessments (in reading and mathematics) in third grade classrooms and its impact 
on instruction, teachers' beliefs, and students' achievement. The purpose of this pape* was 
twofold; (a) to discuss some of the dilemmas and issues that arose in the first two terms of 
work by the authors and participating teachers, and (b) to give a progress report on how 
teachers have changed their instruction and assessment as a result of the project Results of 
the study were "mixed but promising." 

(TC# 000.6DILIST) 



Fort Hays Educational Development Center, The. State Assessment — Math, March 15, 
1995. Available from: Steve Nolte, The Fort Hays Educational Development Center, 
305 Picken Hall, Hays, KS 67601, (913) 628-4382, fax (913) 628-4084. 

Beginning with the 1995 administration, the Kansas math assessment covers problem solving, 
reasoning, and communication skills in grades 4, 7, and 10 Multiple choice, multiple mark, 
and open-ended items are used 

The open-enued problems include many content areas and skills and involve no manipulatives 
Open-ended problems are scored for math communication, reasoning, and problem solving 
using a six-point (^0-5), generalized rubric Problem solving is broken down into four 
dimensions understanding the problem, choosing a p' blem solving strategy, implementing 
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the strategy, and finding/reporting a conclusion. This document includes rubrics, scoring 
forms, and 30 pages of student work in grades 4, 7, and 10 No technical information is 
included. 

(TC# 500.3STAASM) 



Fraser, Barry J.. John A. Malone, and Jillian M. Neale. Assessing and Improving the 
Psychosocial Environment of Mathematics Classrooms. Located in: Journal for 
Research in Mathematics Education 20, 1989, pp. 191-201. 

This article describes the development of a short form of the My Class Inventory to be used in 
sixth g ade math classes to measure the "psychosocial" characteristics of the classroom 
learning environment, eg. social interactions 

(TC# 500.3ASSIMP) 



Glaser, Robert. Expert Knowledge and Processes of Thinking. Located in: Enhancing 
Thinking Skills in the Sciences and Mathematics . Diane Halpern (Ed.), 1992. Available 
from: Lawrence Erlbaum Associates, Publisher, 365 Broadway, Hillsdale, NJ 07642, 
(800) 926-6579. 

In this article the author describes current research on expert performance. Although not 
directly about assessment, the notion of "expertise" can be used to develop criteria for 
evaluating performance tasks. For example experts perceive large, meaningful patterns, have 
skillful self-regulatory processes, etc 

A critical point made by the author is that, "Practice, as it comes about in the usual course of 
training, is not necessarily very efficient. On the basis of our knowledge of the specific 
aspects of competence and expertise, we are able to find ways to compress or shortcut 
experience ." "This is one goal for performance assessment, we help students understand 
current conceptions of the relevant dimensions of a task so that they don't have to rediscover 
this themselves " 

(TC# 050.6EXPKNP) 



Greenwood, Jonathan Jay. On the Nature of Teaching and Assessing “Mathematical 
Power" and “Mathematical Thinking. ” Located in: Arithmetic Teacher. November 
1993, pp. 144-152. 

This is a useful paper for assisting teachers in defining the characteristics of sound 
mathematical thinking 
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The authors discuss ( 1 ) the meaning of the NCTM goals of "mathematical power” and 
•‘mathematical thinking,” (2) seven characteristics of sound mathematical thinking, and (3) 
general rating scales for each characteristic 

The seven diaracteristics are ( 1) Everything you do in mathematics should make sense to 
you (2) Whenever you got stuck, you should be able to use what you know to get yourself 
unstuck ( j ) V ou should be able to identify errors in answers, in the use of materials, and in 
thinking (4) Whenever you do a computation, you should use a minimum of counting 
(5) You should be able to perform calculations with a minimum of rote pencil-paper 
computations (6) W'hen the strategy you are using isn’t working, you should be willing to try 
another strategy instead of giving up (7) You should be able to extend, or change, a problem 
situation by posing additional cor iitions or questions 

No technical information is included Sample instructional tasks and student performances are 
included 

(TC# 500.60NNATT) 



Hall. Greg. Alberta C,rade 9 Performance- Based Assessment — Math, 1992. Available from: 
Greg Hall, Student Evaluation Branch, .Alberta Education, Box 43, 11160 Jasper Ave., 
Edmonton, AB T5K 0L2, Canada. 

The 1992 ninth grade math performance assessment used six stations with hands-on 
activities students circulate through the stations. Testing time for each group of six students 
is 90 minutes The six tasks involved rearranging squares to form different perimeters for the 
same area, measurement and mapping, surface area, collecting and graphing information, 
estimation, and combinations/permutations 

Responses were scored using an analytical trait system having two dimensions: problem 
solving and communication. Each trait was scored on a scale of 0 (totally misunderstood or 
blank) to 3 (readily understood the task, developed a good strategy, carried out the strategy 
and generalized the conclusion). A few possible student responses are included to illustrate 
scoring, but no actual student responses are included. No technical information is included 

(TC# 500.3ALBGRN) 



Halpern, Diane (Ed.). Enhancing Thinking Skills in the Sciences and in Mathematics, 1992. 
Available from: Lawrence Erlbaum Associates, Publishers, 365 Broadway, Hillsdale, 

NJ 07642, (800) 97,6-6579. 

This book is not sipctly about assessment Rather, it discusses the related topics of "What 
should we teach students to do'”' and "How do we do if’" The seven authors "criticize the 
conventional approach to teaching science and math, which emphasizes the transmission of 
tactual information and rote procedures applied to inappropriate problems, allows little 
opportunity for students to engage in scientific or mathematical thinking, and produces inert 
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knowledge and thinking skills limited to a narrow range of academic problems " (p 1 18). In 
general, they recommend that teachers focus on the knowledge structures that students should 
know, use real tasks, and set up instruction that requires active intellectual engagemen* 

The authors give various suggestions on how to bring this about: instructional methods, 
videodiscs, group work, and a host more The final chapter analyzes the various positions and 
raises theoretical issues 

(TC# 500.6ENHTHS) 



Hartman. Charlotte. Mathematical Power Opens Doors, Available from: Vancouver 

School District, PO Box 89,37, Vancouver, WA 98668, (206) 696-7011. 

These open-ended mathematics problems supplement multiple-choice tests of content Pilots 
are being undertaken in several grade levels, however, we only have examples for grade 6 

The document we have contains: ( 1 ) a restatement of the “big” NCTM outcomes (problem 
solving, reasoning, communication, connections), (2) three sample problems, and (3) a sconng 
guide (five traits problem solving, communication, reasoning, math concepts and math 
procedures) A personal communication indicated that materials are used in two phases 
First, several problems are done in a group to model how to proceed Then, students choose 
three of five problems to do individually No technical information or sample student 
performances are included 

(TC# 500.3MATPOO) 



Harvey, John G. Mathematics Testing With Calculators: Ransoming the Hostages. Located 
in: Mathematics Assessment and Evaluation: Imperatives for Mathematics Educators , 
Thomas A. Romberg (Ed.), 1992. Available from: State University of New York Press, 
State University Plaza, Albany, NY 12246. 

This paper looks at the use of calculators in mathematics testing. The premise is that if we 
want students to investigate, explore and discover, assessment must not just measure mimicry 
math. Tests designed to require calculators are more likely to be able to do this. 

Additionally, it is important to incorporate calculators into the curriculum because in the 
technological world of the future, calculators will be essential If we want teachers to use 
calculators in instruction, we need to incorporate them into testing. The author analyzes three 
types of test with respect to calculator use, describes things to consider when designing 
calculator tests, and describes current activity in developing "calculator-active" tests 

(TC# 500.6MATTEC) 
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Herman, Joan L., Davina C. D. Klein, Sara Wakai, et al. Assessing Equity in Alternative 
Assessment: An Illustration of Opportunity-to-Learn Issues, 1995. Available from: 
CRESST-UCLA, 10880 Wilshire Blvd., Suite 700, Los Angeles, CA 90024, (310) 206- 
1532. fax (310) 794-8636. 

This paper w as presented at the April 19, 1995 annual meeting of the American Educational 
Research As 'ociation 

As part of the administration of the 1993 California Learning Assessment System (CLAS), 
samples ot teachers and students were surveyed to examine "opportunity to learn" — the extent 
to which students have equal access to resources, quality instruction, extra-school 
opportunities, and preparation for the CLAS Researchers sampled 13 schools from rural, 
urban, and suburban areas The study included 27 teachers and over 800 students in eiuhth 
grade math classes 

Survey, interview, and classroom observation forms are included Questions include such 
things as use ot calculators, use ot open-ended questions, practice with explaining thinking, 
and attitudes toward math 1 he paper concludes that there are differences among school 
types on some of these factors 

(TC# 500.6ASSEQA) 



Hibbard, K. Michael. Self-Assessment Using Performance Task Assessment Lists, 1994. 
Available from: Region 15 School District, PO Box 395, Middlebury, CT 06762, 

(203) 758-8250. 

Pomperaug Regional School District 1 5 (Middlebury, CT) has developed performance criteria 
tor a wide variety of student products and performances. These criteria, called “task 
assessment lists are written to be developmentally appropriate for various grade levels 
(primary, elementary, and secondary) The lists are used to communicate to students the 
attributes of quality products and performances, promote student self-assessment and self- 
reflection, and grading. This document contains sample task assessment lists for graphs 

(TC# 500.3SELUSP) 



Horn, Kermit, and Marilyn Olson. 1992-1993 Lane County Fourth Annual Project Fair. 
Official Guidelines, Criteria & Registration Forms for Grades K-12. Available from: 
Kermit Horn or Marilyn Olson, Project Fair Coordinators, Lane Education Service 
District, Instructional Services Division, PO Box 2680, Eugene, OR 97402, 

(503) 689-6500. 

This handbook is given to students (grades K-12) interested in registering for the Lane County 
project fair. It contains information on registration, criteria by which projects will be judged, 
and help with getting started The document also gives some excellent ideas on 
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interdisciplinary projects. No samples that illustrate score points on criteria are included; the 
criteria, although an excellent start, are still a little sketchy. 

(TC# 000.3LANCOP) 



Hynes, Michael C. A>5 Mathematics Program Evaluation - A Florida Model Curriculum 
Project; School Handbook, 1993. Available from: University of Central Florida, College 
of Education, Orlando, FL 32816, (407) 823-6076. 

This handbook is intended to provide schools with the tools to assess K-5 mathematics 
programs in terms of student outcomes, program goals, curriculum, learning environment, 
learning resources, program support, evaluation, and equal access The handbook includes 
sample instruments for each area Most of these are surveys except for assessment of student 
outcomes 

Seventeen sample performance tasks (eight for primary and nine for grade 5) are included to 
assess various student outcome goals. These are open-ended (there is more than one correct 
answer) and most require explanations of and rationale for, procedures used. Tasks do not 
use manipulatives — all are written/visual. All tasks are done individually; there is no group 
collaboration Performance is rated using a generic four-point scale — "Exemplary (Level A)” 
to "No Attempt (Level E).” /\n E/:emplary response is one which: “Provides excellent 
responses in all areas of problem solving, reasoning, communication, connections, and 
mathematics content. Goes beyond the excellent category. Shows creative thinking, elegant 
communication, and/or unique approaches to the problem Uses additional information to 
extend the solution.” A sample student response for each score point is included to illustrate 
the scoring procedure 

The handbook includes a disk containing each of the instruments, a "School Report Card” and 
the scoring rubric No technical information is included The author has granted permission 
for educators to reproduce materials for use with students. 

(TC# 500.3K-5MAP) 



Illinois State Board of Education. Effective Scoring Rubrics — A Guide to Their 

Development and Use, 1995. Available from: Dlinois State Board of Education, Dept, of 
School Improvement Services, School and Student Assessment Section, 100 N. First St., 
Springfield, EL 62777, (217) 782-4823, fax (217) 784-6125. 

This short booklet provides a good overview of the characteristics of sound performance 
criteria (rubrics, scoring guides) and uses Illinois' writing and math rubrics as examples The 
author values analytic/holistic, developmental, generalizable rubrics for the classroom 

(TC# 150.6EFFSCR) 
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Illinois State Board of Education. Performance Assessment in Mathematics: Approaches to 
Open-Ended Problems, 1995. Available from: Illinois Educational Service Center Sii, 
1819 W. Pershing Rd., 4C(s), Chicago, IL 60609. 

This document is a very concise and well written guide for assessing problem solving using 
open-ended problems The document includes a nice definition of problem solving, criteria for 
writing open-ended problems (with samples illustrating different types), a sample three-trait 
rubric (math knowledge, strategic knowledge, and communication) for scoring student work, 
and 41 scored samples No technical information is included 

(TC# 509.6PEROPE) 



•Johnson, Judi .Mathis. Portfolio Assessment in Mathematics: Lessons from the Field. 
Located in: The Computing Teacher 21, March 1994, pp. 22-23. 

The author describes the results of a study in which she interviewed 20 successful math 
teachers. Her major conclusion is that math, assessment, and technology should not be 
competitors for teachers attention, but components of a successful math classroom. 
Portfolios are used to both document and promote student learning and student responsibility 
tor learning through self-reflection. Technology is used to learn concepts, produce work for 
the portfolios, and to produce the portfolio itself Assessment is used to guide planning and 
influence instruction 

(TC# 500.6PORMAL) 



Katims, Nancy. PACKETS* Program: An Illustration of Classroom-Based Alternative 
Assessment. Available from: Director, Office of School Services, Educational Testing 
Service, Princeton, NJ 08541, (609) 921-9000, fax: (609)734-5410. 

The author describes the P ACKETS* mathematics performance assessment exercises for 
grades 6-8. under development by Educational Testing Service (ETS). The document 
includes the rationale for the approach taken, two extended examples with samples of student 
work, and suggestions on how to use the samples in the classroom. 

PACKETS* activities try to pose a "big ideas" math problem in the context of developing a 
product for a client. Students read a context-setting newspaper article, discuss a set of 
readiness questions, and work together in groups of three for about two class periods 
Students write-up their solutions and present them to the class. Students have the opportunity 
to revise after initial evaluation. Scoring is based on teacher observations (of group work and 
mathematical thinking) during the process of doing the activity, analyzing the math used, and 
evaluating the product for mathematical appropriateness, reasonableness of the solution, and 
appropriateness for the stated purpose. The document contains no scored student work and 
no tef'hnical information 

(TC# 500.3PACKET) 
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Kentucky Department of Education. Kentucky Instructional Results Information System 
(KIRIS), 1993-94. Available from: Kentucky Department of Education, Division of 
Accountability, 1900 Capital Plaza Tower, 500 Mero St., Frankfort, KY 40601, 

(502) 564-4394. 

The Kentucky Instna nonal Results Information System is an assessment program that 
monitors student achitvement (grades 4. s, and 12) in reading, social studies, science, and 
mathematics The assessment has three parts multiple choice/short essay, performance 
assessment, and portfolio: .\ssessment results place students in one of four performance 
levels novice, apprentice, proficient, or distinguished. The document we received contains 
grade 4. 8. and 12 performance assessment items in reading, social studies, science, and 
mathematics All items are paper and pencil Task-specific scoring guides are included 

( TC# 060.3K1RJS94) 



Khattri, Nidhi. Performance Assessments: Observed Impacts on Teaching and Learning, 
1995. Available from: Pelavin Associates, 2030 M St. NW, No. 800, Washington, DC 
20036. 

The 'e , attempted to document the impact of performance assessment on teaching and 
learnir The author visited 14 schools in fall 1994 and spring 1995 to examine student work; 
obser m classrooms, and interview school personnel, students, and parents. 

1 he two strongest conclusions to be drawn from these findings are that. (1) students are being asked to 
w nte. to do project-based assignments, and to engage in group learning due to the use of performance 
assessments; and (2) as a result of project -based assigrunents. students are more motivated to learn. 
Furthermore, because of the use of performance-based assignments and the degree of freedom accorded to 
students in shaping their own work collaboration also is evident. Increasingly, teachers are viewing 
students as active learners 

All of the observed and reported effects, it must be emphasized, were mediated to a large degree by; (a) the 
form of the assessment (e g., portfolio or performance event); (b) the degree of integration of the 
assessment into the classroom; and (c) the level of support provided to incorporate the assessment into 
routine classroom activities. The positive and intended effects on pedagogy are most evident for sites 
engaged in portfolio assessments, mostly because the portfolio format provides teachers and students 
control over products coupled with a structure for documenting student work and student progress on an 
ongoing basis. 

The author states: "Performance assessments, thus, remain a lever for reform, but what 
exactly is to be leveraged still remains to be defined " 

(TC# 150.6PERASO) 
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Kloosterman, Peter, and Frances K. Stage. Measuring Beliefs About Mathematical Problem 
Solving. Located in: School Science and Mathematics 91 . Marrh 199 ?. pp ino.iK 

The authors describe the development of a scale to assess student beliefs about mathematics 
The instrument, designed for grades 7+, is called the Indiana Mathematics Beliefs Scale. 
Thirty- six questions cover six beliefs: (1)1 can solve time-consuming mathematics problems 
(2) There are word problems that cannot be solved with simple, step-by-step procedures. 

) Understanding concepts is important in mathematics (4) Word problems are important in 
mathematics (5) Effort can increase mathematical ability. (6) Mathematics is useful in dailv 
life The paper includes technical information based on studies with college students 

(TC# 500.3MEABEM) 



Knight, Pam. How I Use Portfolios in Mathematics. 1992. Located in: Educational 
Leadership 49, pp. 71-72. Also available from: Twin Peaks Middle School, Poway 
Unified School District, 14012 Valley Springs Road, Poway, CA 92064. 

The author describes her first year experimentation with portfolios in her middle school 
algebra classes She had her students keep all their work for a period of time and then sort 
through it to pick entries that would best show ( 1 ) effort and learning in algebra, and 
(2) activities that had been the most meaningfiil There is some help with what she did to get 
started, and discussion of the positive effects on students There is some mention of 
performance criteria, but no elaboration One student self-reflection is included, but no 
technical information 

(TC# 530.3HOWIUS) 



Koretz, Daniel, Daniel McCaffrey, Stephen Klein, et al. The Reliability of Scores from the 
1992 Vermont Portfolio Assessment Program — Interim Report. December 1992. 

Available from: RAND Institute on Education and Training, National Center for 
Research on Evaluation, Standards, and Student Testing, UCLA Graduate School of 
Education, 10880 Wilshire Blvd., Los Angeles, CA 90024, (310) 206-1532. 

Beginning in 1990, RAND has been evaluating Vermont's portfolio assessment program. This 
paper reports on the study conducted during school year 1991-92. Basically, RAND found 
that interrater agreement on portfolio scores was very low for both writing and math. The 
authors speculate that this resulted from aspects of the scoring system, aspects of the 
operation of the program, and the nature and extent of training raters This report provides 
good advise and caution for others setting up portfolio systems for large-scale assessment. 

(Note; A paper published in 1995 reports that with better training, scoring becomes more 
reliable.) 

(TC# 150.6RELSCV) 
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Koretz, Daniel, Brian Stecher, and Edward Deibert. The Vermont Portfolio Assessment 
Program: Interim Report on Implementation and Impact, 1991-92 School Year. 

Available from: RAND Institute on Education and Training, National Center for 
Research on Evaluation, Standards, and Student Testing, UCLA Graduate School of 
Education, 10880 Wilshire Blvd., Los Angeles, CA 90024, (310) 206-1532. 

Beginning in 1990. RAND has been carrying out a multi-faceted evaluation of Vermont's 
portfolio assessment program This paper reports on questionnaires and interviews conducted 
during school years 1990-91 and 1991-92 Results indicated that 

1 There was a significant impact on instruction, but teachers felt somewhat confused about 
w hat they were supposed to do 

2 The portfolios took a lot of classroom space and tended to be viewed by teachers as an 
add-on rather than as "the" instruction 

3 Teachers felt they knew more about students as the result f doing portfolios 

4 Students had some difficulty doing portfolio problems 

5 Reported effect on low achieving students was mixed 
(TC# 150.6VERPOP) 



Koretz, Daniel, Brian Stecher, Stephen Klein, et al. The Vermont Portfolio Assessment 
Program: Findings and Implications. Located in: Educational Measurement: Issues 
and Practice 13, Fall 1994, pp. 5-16. 

The authors describe the Vermont portfolio assessment system, summarize key findings from 
a series of evaluations of the Vermont system, and discuss the implication of the findings for 
performance assessment in general. Although somewhat technical, the paper clearly points 
out the reliability and validity concerns surrounding the use of portfolios for large-scale 
assessment, and the tensions between using portfolios for large-scale assessment and using 
portfolios for instructional improvement. In short, portfolios are not magic and they have to 
be thoughtfully implemented. The paper presents a very thoughtful discussion of the issues. 

(TC« 000.6VERPOF) 



Kulm, Gerald. (Ed.) Assessing Higher Order Thinking in Mathematics, 1990. Available 
from: American Association for the Advancement of Science, 1333 H Street NW, 
Washington, DC 20005, (301) 645-5643. 

This book contains a series of articles that address various topics in mathematics assessment 
The articles address three broad topics 
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1 The rationale for assessing mathematics problem solving and the need to have 
assessment devices that reflect this emphasis. 

2 Issues that come up when trying to assess higher-order thinking skills in mathematics 

3 General discussions of what to assess and how to assess it. 

There are a few examples of actual assessment techniques The most relevant articles are 
included on this bibliography as separate entries 

(TC# 500.6ASSHIO) 



Kulm, Gerald. Mathematics Assessment — What Works in the Ciassroom, 1994. Available 
from: Jossey-Bass Inc., Publishers, 350 Sansome St., San Francisco, CA 94104. 

This excellent book provides a good background for changes in assessment, has a great 
chapter on scoring rubrics, and provides good case studies of classroom alternative 
assessment use in grades 4-12 

(TC# 500.6MATASW) 



Lambdin, Diana V. and Vicki L. Walker. Planning for Classroom Portfolio Assessment. 
Located in: Arithmetic Teacher . February 1994, pp. 318-324. 

The authors report on the development of a portfolio system for their secondary mathematics 
students (This was part of a larger project for all grade 3-12 teachers ) The authors have 
students assemble a “permanent portfolio” from a "working portfolio ” The working portfolio 
is a place to store all potential permanent portfolio entries. The permanent portfolio is 
assembled from the working portfolio The goals are to promote student self-assessment, 
improve communication with students and parents, and gain a broader picture of the student 
than available with traditional tests. The goal is not to assess the breadth of student 
knowledge — this is done with traditional methods. The authors include task guidelines that 
are given to students. 

In addition to student and peer evaluation of portfolio selections and the portfolio as a whole, 
the teacher has guidelines for grading based on diversity of selection, quality of written 
reflections about selections, and portfolio organization. The authors present a little detail that 
flesh out these criteria. The document includes some student work but no technical 
information 

(TC# 500.3PLAPOA) 
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Lane, Suzanne. QUASAR Cognitive Assessment Instrument, (QCAI), 1993. Available from: 
QUASAR (Quantitative Understanding: Amplifying Student Achievement and 
Reasoning), Learning Research & Development Center, University of Pittsburgh, 

3939 O’Hara St., Pittsburgh, PA 15260, (412) 624-7791. 

The QCAI (QUASAR Cognitive Assessment Instrument) is designed to measure long-term 
growth of students in math thinking and reasoning. Information for this review was taken 
from several different papers (all of which are bound together in our collection) 

Thirty-three tasks were designed for sixth and seventh graders The tasks were designed to 
provide a good sample of math thinking and reasoning skills by using a variety of 
representations, approaches and problem strategies. Students were asked to provide a 
justification for a selected answer or strategy, explain or show how an answer was found, 
translate a problem into another representation (picture or equation), pose a mathematical 
question, interpret provided data, and extend a pattern and describe underlying regularities. 

The tasks were field-tested. General descriptions for all the tasks and details on a few 
individual tasks are provided in these materials 

Scoring is done via a generalized holistic, four-point rubric which directs raters to consider 
mathematical knowledge, strategic knowledge and communication (Each of these dimensions 
is laid out very clearly and could be used as the basis of an analytical trait scoring scale ) The 
generalized rubric is then tailored to individual problems by specifying features of responses 
that would fall at different scale points The generalized scoring guide is included in these 
materials but not the task-specific adaptations 

(TC# 500.3QUACOA) 



Larter, Sylvia. Benchmarks: The Development of a New Approach to Student Evaluation, 
1991. Available from: Toronto Board of Education, 155 College St., Toronto, 

ON M5T 1P6, Canada, (416) 598-4931. 

Benchmarks are student performances on tasks tied to Provincial educational goals. Each 
Benchmark activity lists the goals to be addressed, the task, and the scoring system. Tasks 
vary considerably Some require very discrete responses (e g., knowledge of multiplication 
facts), while others are more open-ended. There are 129 Benchmarks developed in language 
and mathematics for grades 3, 6. and 8 

For many of the tasks, a general, holistic, seven-point scale ("no response" to "exceptional 
performance [rare]") was used as the basis to develop five-point, task-specific, holistic scoring 
scales For other tasks, scoring appears to be right/wrong. Holistic scoring emphasizes 
problem solving, method of production, process skills, and accuracy, although students can 
also be rated on perseverance, confidence, willingness, and prior knowledge, depending on the 
Benchmark. 

The percentage of students at each score point (e g , 1-5) is given for comparison purposes, as 
are other statistics (such as norms) when appropriate Anchor performances (e g., what a "3" 
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performance looks like) are available either on video or in hard copy. This report describes 
the philosophy behind Benchmarks, how they were developed, and a few of the specific 
Benchmarks. Some technical information is described (factor analysis, rater agreement), but 
no student performances are provided 

(TC# 100.6BENCHM) 



Lawrence, Barbara. Utah Core Curriculum Performance Assessment Program: 
Mathematics, 1993. Available from: Profiles Corporation, 507 Highland Ave., 

Iowa City, lA 52240. 

The Utah State Office of Education has developed 90 constructed-response items in 
mathematics, science and social studies (five in each of grades 1-6 for each subject) to 
complement multiple-choice tests already in place .Assessments are designed to match the 
Utah Core Curriculum goals Although districts are required to assess progress toward these 
goals, the state-developed assessments are optional. 

The mathematics assessments are designed to measure logical reasoning, number meanings, 
number operations, number representation, computation, estimation, algebra, data sets, 
probability, geometry', measurement, fractions, and patterns Each task has several questions 
relating to a theme For example, a sixth grade task called “Lab Tech” has students do such 
things as complete a number sequence (adding milliliters to a chemical solution every 10 
minutes), and ‘ You need to plant two kinds of seeds. You must have 12 pots of one kind of 
seeds and 18 pots of the other kind of seeds. You need to plant the same total number of 
each kind of seed What is the least number of each kind of seed you could plant?” 

Scoring is task-specific and based on the degree of correctness of the response, For example, 
in the “Lab Tech" example, the student gets 3 points if he or she correctly completes the 
entire sequence Points are totaled for each task and between tasks for each of the skill areas 
being assessed. Four levels of proficiency on each skill are identified; advanced, proficient, 
basic and below basic Cut scores for each level are based on percent correct (approximately 
90%=advanced, 70%=proficient, 40%=basic, below 40%=below basic) and behavioral 
descriptions of performance at eacli level. 

Assessment activities are bound in books for each grade level/ subject. Each task includes 
teacher directions, student test-taking materials, and scoring guides. The Office of Education 
has collected information on teacher reaction to the assessments from the field test. No other 
technical information is available at this time. A training video is available which helps 
teachers use the assessments. 

( TC# 500.3UTACOC AND 000.6INTIJTCv— video) 



NWREI,, September 1995 
rest Center, (503) 275-9582 



32 



3 (.‘ 



Mathematics 



Leach, Eilene L. An Alternative Form of Evaluation that Complies with NCTM's Standards. 
Located in: The Mathematics Teacher 85, November 1992, pp. 628-632. Also available 
from Centaurus High School, 10300 S. Boulder Rd., Lafayette, CO 80026. 

This teacher uses scored discussions to assess and promote problem solving, communicating 
mathematically, and group process skills in her high school math classes. She has three to six 
students face each other in front of the rest of the class and spend about five minutes trying to 
solve a problem Individuals can earn positive points for such things as "determining a 
possible strategy to use," "recognizing misused properties or arithmetic errors," or "moving 
the discussion along " They can earn negative points by doing such things as: "not paying 
attention or distracting others." and "monopolizing." 

The article has a thorough discussion of how the teacher sets up the classroom, introduces the 
procedure to students, scores the discussion, and handles logistics The author also discusses 
the positive effects this procedure has had on students, and the additional insight she has 
obtained about her students 

.All scoring is teacher-centered, but it wouldn't necessarily have to be No technical 
information is included 

(TC# 500.3ALTFOE) 



Lehman. Michael. Assessing Assessment: Investigating a Mathematics Performance 
Assessment, 1992. .Available from: The National Center for Research on Teacher 
Learning, 116 Erickson Hall, Michigan State University, East Lansing, Ml 48824-1034. 

This monograph, by a high school math teacher, describes his attempt to develop a better 
method of assessing algebra problem solving, concepts, and skills than traditional paper-and- 
pencil tests. The assessment technique involves giving students problems to solve as a group, 
and then having them explain their results in front of a panel of judges. Three examples )f 
problems are provided, as is a brief description of the scoring criteria (making sense of the 
problem, and problem-solving strategies), accuracy of results, interpreting results, ability to 
communicate results, and an explanation of what they did. However, these criteria are not 
elaborated on, and, although samples of student explanations are provided, these are used to 
describe the understandings the teacher reached about his students, not to anchor the 
performance criteria. 

The author also provides a brief summary of the strategies he uses to help students develop 
greater depth in their understanding of algebraic principals and their interrelationships — small 
group cooperative learning, requiring justifications of approaches, etc. 

(TC# 530.3ASSASO 
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Lehman, Michael. Performance Assessment — Math, 1992. Available from: Michael 
Lehman, Holt Senior High School, 1784 Aurelius Rd., Holt, MI 48842, (517) 694-2162. 

This paper is related to the one above, and provides additional information Students are 
given six problems (some having only one right answer and some having more than one right 
answer) to solve as a team (four students per team) The team then spends an hour with a 
panel ot three judges Judges can ask any student to explain the team's solution and problem- 
solving strategy on any of the six problems (Therefore, dll students must have knowledge of 
all six problems ) Then the judges assign the team a new problem to w ork on while they 
watch 

Student responses are scored on making sense of the problem, solution strategies, accuracy 
ot results, ability to communicate results, ability to answer questions posed by the judges, 
three judgments of group-process skills and an overall judgment of student understanding 

A complete set often tasks (six pre-assigned, and four on-the-spot) are included for 
Algebra II The scoring guide and a few sample pre-calculus projects are also included N'o 
technical information nor sample student performances are included. 

(TC# 500.3PERASM) 



Lesh, Richard. Computer-Based Assessment of Higher Order Understandings and Processes 
in Elementary Mathematics. Located in: Assessing Higher Order Thinking in 
Mathematics, Gerald Kulm (Ed.), 1990. Available from: American Association for the 
Advancement of Science, 1333 H Street NW, Washington, DC 20005, (301) 645-5643. 

This article is as much about how meaningful learning occurs and the nature of the structure 
ot knowledge in mathematics, as it is about use of computers in math instruction and 
assessment The basic premise is that computer-based tests should not simply be pencil-and- 
paper tests delivered on-line. They should be part of an integrated instruction and assessment 
system that supports both learning facts and developing the meaningful internal structuring of 
these facts to form a coherent knowledge system. 

The article discusses three things 

1 Principles underlying a modeling perspective of learning and assessment (ideas such as 
learmng and problem-solving situations are interpreted by the learner by mapping them 
to internal models, and several "correct" alternative models may be available to interpret 
a given situation) 

Five objectives that should be emphasized in K-12 math (such as going beyond isolated 
bits of knowledge to construct well-organized systems of knowledge, and think about 
thinking) 

J Specific types of assessment items that can be used to measure these deeper and broader 
understandings (such as conceptual networks and interactive word problems) 
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Many sample problems are provided 
(TC# 500.6COMBAA) 



Lesh, Richard and Susan J. Lamon, Eds. Assessment of Authentic Performance in Schooi 
Mathematics, 1992. Available from: American Association for the Advancement of 
Science, 1333 H St. .NW, Washington. DC 20005. 

This book contains articles by several authors on the topics of: assessment objectives — what 
should we assess, examples of assessments, classroom assessment, and scoring and reporting 

( TC# 500.6 ASS A UP) 



Lester, Frank K, Jr. An Assessment Model for Mathematical Problem Solving. Located in: 
Teaching Thinking and Problem Solving 10, September/October, 1988, pp, 4-7. Also 
available from: Lawrence Erlbaum Associates, Inc., Journal Subscription Department, 
365 Broadway, Hillsdale, NJ 07642, (800) 962-6579 

This article presents a model for assessing both the problem-solving performance of students 
and assessing the task demands of the problem to be solved. The dimensions of problem 
solving (which could be used as a scoring rubric) are understanding/formulating the question 
in a problem, understanding the conditions and variables in the problem, selecting the data 
needed to solve the problem, formulating subgoals and selecting appropriate solution 
strategies to pursue, implementing the solution strategy and attaining subgoals, providing an 
answer in terms of the data in the problem, and evaluating the reasonableness of an answer 
The article describes these in some detail 

The problem features that can affect a student's success in solving a problem are; the type of 
problem, the strategies needed to solve it, the mathematical content/types of numbers used, 
and the sources from which data need to be obtained to solve the problem 

(TC# 500.3ANASSM) 



Lester, Frank K. Jr., and Diana Lambdin-Kroll. Assessing Student Growth in Mathematical 
Problem Solving. Located in: Assessing Higher Order Thinking in Mathematics . 

Gerald Kulm (Ed.), 1990. Available from: American Association for the Advancement 
of Science, 1333 H Street NW, Washington, DC 20005, (301) 645-5643. 

The authors present a model of factors that influence problem-solving performance, and 
discuss several problem-solving assessment techniques 

A good assessment program in math should collect information about toe following; affect 
(attitudes, preferences, and beliefs), and cognitive/processes ability to get the right answer 
(both whether they get the right answer, and the strategies used). The program should also 



NWREL. September 
Test Center, (503) 275-9582 



Mathematics 



systematically define and cover the features of tasks (problem type, math content, required 
strategies, etc.) since these affect performance and should be reflected in instruction ^ 

In order to gather information on these three categories of factors, the authors briefly review 
observations, interviews, student self-reports, and holistic and analytic scoring of 
pertbrmances They recommend against multiple-choice questions 

This paper is a general theoretical discussion, no actual tasks, problems or scoring guidelines 
are provided 

(TC# 500.6.ASSSTG) 



Linn, Robert L., Eluabeth Burton, Lizanne DeStefano et al. Generalizability of Sew 
Standards Project— 1993 Pilot Study Tasks in Mathematics, 1995. Available from: 

Center for Research on Evaluation, Standards, and Student Testing, School of 
Education, University of Colorado, Campus Box 249, Boulder, CO 80309, 

(303) 492-8280. 

The authors report on a study in which they examined the ability to draw general conclusions 
about student skills from a small sample of performances They repeat the finding that more 
error is introduced into assessment results from the sample of ta.sks the student gets than from 
Inconsistencies between raters when scoring In other words, it is difficult to infer general 
student skill level from performance on a small number of tasks because student performance 
is not \ ery consistent from task to task 

(TC# 150.6GENNES) 



.Vladaus, George F. and Thomas Kellaghan. The British Experience with 'Authentic' 
Testing. Located in: Phi Delta Kannan 74, February 1993, pp. 458-459, 462-463, 
466-469. 

This paper presents a good summary of the performance assessment approach in England and 
results of the age 7 assessment in math, science, and language arts 

• technical problems • high cost 

• lots of sti ess • hard to manage 

Nevertheless, there was value, and most staff felt it gave them food for thought about 
instructional practice England is retreating from its approach 

(TC# 000.6BRIEXA) 
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Marshall, Sandra P. Assessing Knowiedge Structures in Mathematics; A Cognitive Science 
Ftrspective. Located in: Cognitive Assessment of Language and Mathematics 
Outcomes. Sue Legg & James Algina (Eds.), 1990. Available from: Ablex Publishing 
Company, 355 Chestnut St., Norwood, NJ 07648. 

This article discusses the implications of recent advances in cognitive science for mathematics 
assessment The goal in using this research to develop assessment techniques is to determine 
the extent to which students have acquired specific cognitive skills rather than merely whether 
they can correctly solve particular problems 

Cognitive theory holds that people solve problems by using three knowledge structures — 
declarative (facts), procedural (algorithms and production rules), and schema (frames that 
relate facts and production rules) To solve a problem, a person must first find the right 
schema, must then correctly implement a set of production rules, and must have stored 
correctly the facts and knowledge required to carry out the necessary algorithms specified by 
the production rules Errors can occur in any of these three areas 

Researchers are currently engaged in specifying these knowledge structures in such detail that 
they can develop computer simulations that can, first, solve problems, and second, reproduce 
student errors by leaving out or altering various parts of the necessary structures In this way, 
errors in student responses can be tracked back to the erroneous structure used The author 
specifically mentions work in the area of simple arithmetic operations, geometry, and word 
problems 

Additionally, the author discusses two other ways of assessing these things in students — 
reaction time (to assess how automatic a function is), and multiple-choice problems (e g., 
"which of the following problems can be solved in the same way as the one stated above?" to 
get at schema knowledge). Some time is spent with multiple-choice problems to explore 
various types of problems and the technical issues that arise with them. 

It should be pointed out that all these procedures are experimental; none have progressed to 
the point where there is a final product that can be ordered and installed. 

(TC# 500.6ASSKNS) 



.Marshall, Sandra P. The Assessment of Schema Knowledge for Arithmetic Story Problems: 

A Cognitive Science Perspective, 1990. Located in: Assessing Hiaher Order Thinking in 
Mathematics . Gerald Kulm (Ed.). Available from: American Association for the 
Advancement of Science, 1333 H Street NW, Washington, DC 20005, (301) 645-5643. 

The Story Problem Solver (SPS) was created to support instruction based on a theory of 
memory architecture called schemata Under such theories, human memory consists of 
networks of related pieces of information Each network is a schema — a collection of well- 
connected facts, features, algorithms, «kills, and/or strategies 
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Adult students are explicitly taught five problem-solving schemas and how to recognize which 
schema is represented by a story problem. SPS is a computerized assessment method in which 
several different item types are used students pick out the schema or general solution 
strategy that fits a given storv' problem, decide whtch information in the story problem fits into 
the various frames of the schema, ideritify the steps needed to solve a problem, and decide 
whether the necessary information is given in the problem 

Some ot the schema shells and item types are given as examples No technical information is 
included 

( TC# 500.3ASSOFS) 



.Maryland Assessment Consortium. Information Packet, 1993-94. Available from: Jay 
McTighe, Maryland Assessment Consortium ^Frederick County Public Schools, 

115 E. Church St., Frederick, MD 21701, (301) 694-1337, fax (301) 694-1800. 

This handout contains an overv iew of the Maryland Assessment Consortium and two sample 
elementary assessment tasks The first is an integrated task (social studies, science, and 
writing) which requires students to compose an "Aesop’s Fable” after reading and analyzing 
such fables and discussing where they come from The second is a math activity on planning a 
backpacking trip Some scoring guides are task specific and others are generalized. Task- 
specific scoring tends to be used for knowledge questions and generalized scoring tends to be 
used for big outcomes such as problem solving No student work or technical information 
is included 

(TO¥ 000.3MARASC) 



Maryland Assessment Consortium. Performance Assessment Tasks Elementary Level, 
Volume 6 and Performance Assessment Tasks Middle School, Volume 7, 1994-95. 
Available from: Jay McTighe, Maryland Assessment Consortium (^Frederick County 
Public Schools, 1 15 E. Church St., Frederick, MD 21701. (301) 694-1337, 
fax (301) 694-1800. 

The Maryland Assessment Consortium has published two notebooks of sample performance 
tasks in language arts, science, social studies, and mathematics. Some tasks are integrated 
across these areas. Volume 6 contains 13 elementary tasks and Volume 7 contains 9 middle 
school tasks. Each task includes complete instructions, test booklets and scoring guide, 
extension activities, special education modifications, and references Many tasks require 
several steps or portions related to a theme, group work, hands-on activities, reading and 
interpreting materials, and writing in subject matter areas Performance criteria tend to be 
task-specific with separate criteria for each part of the task 
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■All tasks have been pilot-tested No technical information nor sample student work are 
included. This document includes one sample elementary task and one sample middle school 
task Full sets are available from the author 

(TC# 000.3PERAS1, 



Maryland Department of Education. Maryland School Performance Assessment Program, 
1992. Available from: Gail Lynn Goldberg, Maryland Department of Education, 
Maryland School Performance Assessment Program, 200 W. Baltimore St., Baltimore, 
MD 21201,(410)333-2000. 

Maryland has released six performance tasks that illustrate the 1992 assessment. This review 
is based on three of them, one task at each of grades 3, 5 and 8 The tasks are integrated 
across subject areas and use some combination of information and skills in science, math, 
writing, reading, and social studies The three tasks we have relate to the weather (Grade 3), 
snowy regions of the country (Grade 5) and collisions (Grade 8). Each task has both 
individual and group work and proceeds through a series of exercises that require reading, 
designing and conducting experiments, observing and recording information, and writing up 
results. 

Student responses are scored using two basic approaches: generalized holistic or analytical 
trait scoring for the "big" outcomes such as communication skills, problem solving, science 
process skills, and reasoning, and specific holistic ratings of conceptual knowledge and 
applications For example, the task on collisions is scored both for knowledge of the concepts 
of mass and rate/distance, and for general science process skills (collecting and organizing 
data, and observation) and communication skills Thus, some scoring guides are generalized 
across tasks, and some list specific features from individual tasks to watch for 

The materials we have allude to anchor performances and training materials, but these are not 
included in our samples. Neither information about student performance, nor technical 
information about the tests is included. 

(TC# 060.3MARSCP) 



Mary land State Department of Education. Maryland School Performance Assessment 
Program: Resource Library — MSPAP Public Release Tasks: Planning a Zoo, Grade 3, 
Mathematics; School Fair, Grade 5, Matherruttics; and Birth Dates, Grade 8, 
Mathematics; July 1994. Available from: Maryland State Department of Education, 
Maryland School Performance Assessment Program, 200 W. Baltimore St., 

Baltimore, MD 21201, (410) 333-2000. 

This document contains a 1994 released math assessment booklet and scoring guide for each 
of grades 3, 5, and 8 Booklets consist of a scenario around which several related math 
activities are structured. For example, at grade 8 there are several questions related to the 
theme of birthdays Some questions require only a short answer while others require more 
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complex responses, e g., drawing a graph, writing an explanation of an answer, or making 
predictions. Scoring is task-specific 

The document contains tasks and scoring guides complete with sample student responses No 
technical information is included 

(TC# 500.3MSPAP.VI) 



.Maryland State Department of Education. Scoring MSPAP (Maryland School Performance 
Assessment Program): A Teacher’s Guide, 1993. Available from: Gail Lynn Goldberg, 
.Maryland Department of Education, Maryland School Performance Assessment 
Program, 200 W. Baltimore St., Baltimore, MD 21201, (410) 333-2000. 

This document presents information about the 1993 MSP.AP in grades 3, 5, and 8 
philosophy, general approach, sample tasks, and performance criteria. There are sample tasks, 
performance criteria and student responses for the following areas: expository, persuasive 
and expressive writing, reading comprehension, math, science, and social studies. 

Scoring can be done three different ways depending on the task: generalized scoring rubrics 
that can be used across tasks (e g., persuasive writing); generalized scoring rules that are not 
‘ as detailed as rubrics (e g , language usage); and scoring keys that are task-specific (e g., 
many math tasks are scored for the degree of "correctness" of the response) 

No technical information is included 

(JCU 000.3SCOMST) 



.Maryland State Department of Education. Teacher to Teacher Talk: Student Performance 
on MSPAP (Maryland School Performance Assessment Program), 1992. Available from: 
Gail Lynn Goldberg, Maryland Department of Education, Maryland School 
Performance Assessment Program, 200 W. Baltimore St., Baltimore, MD 21201, 

(410) 333-2000. 

This publication presents teacher reactions to their experience of scoring performance 
assessment tasks on the 1992 Maryland School Performance Assessment Program (MSPAP) 
The MSPAP covered reading, writing, math, social studies and science in grades 3, 5s and 8. 
Comments are organized by grade and subject Most comments have to do with two topics: 
what teachers learned about students as the result of participating in the scoring, and how the 
performance tasks should be revised 

(TC# 000.6TEATET) 
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Manano, Robert J., Debra J. Pickering, Jo Sue Whisler, et al. Authentic Assessment, 
undated. Available from: Mid*Continent Regional Laboratory (McREL), 

2550 S. Parker Rd., Suite 500, Aurora, CO 80014, (303) 337-0990. 

This document appears to be a series of handouts used in training Although not specifically 
about math, the document does discuss some "big" outcomes related to math such as complex 
thinking, information processing, communication, etc 

Materials include definitions of assessment terms, a procedure for developing performance 
assessment tasks, and samples of tasks and scoring guides. The general approach is mix and 
match — tasks are meant to elicit several target behaviors on the part of students which are 
then scored with generic performance criteria For example, a problem-solving task requires 
students to draw a picture of their neighborhoods without using any circles or squares. 
Performances are scored for knowledge (geometry), complex thinking (ability to identify 
obstacles in the way of achieving desired outcomes), and effective communication (ability to 
express ideas clearly) 

Sample tasks are in the areas of science, math and social studies There are general mix-and- 
match scoring guides for: Knowledgeable Person, Complex Thinker, Information Processor, 
Effector Communicator/Producer, Self-Directed Learner, and Collaborative Worker. Scoring 
guides are generally not very descriptive. For example, one of the three traits included in the 
scoring guide for Skilled Information Processor is "effectively interprets and synthesizes 
information." To get a "4" (the highest score possible) the student "consistently interprets 
information gathered for tasks in accurate and highly insigi'.tful ways and provides synthesis of 
that information that are highly creative and unique " This is 'sically just a restatement of the 
trait title 

The authors have begun to develop a useful approach to performan-^e assessment (mix-and- 
match tasks and performance criteria), but the criteria need to be filled out a little more 

(TCU 150.6AUTASS) 



Massell, Diane. Setting Standards in Mathematics and Social Studies. Located in: 
Education and Urban Society 26, February 1994, pp. 118-140. 

This article describes, analyzes, and contrasts two efforts to set curriculum standards — the 
National Council of Teachers of Mathematics (NCTM) efforts in math, and the state of 
California’s efforts in history-social studies It describes the history of development and 
provides ideas on what it takes to have a successful development effort. 

(TC/^ 000.5SETSTM) 
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Mathematical Science Education Board, National Research Council. Measuring Up — 
Prototypes for Mathematics Assessment, 1993. Available from: National Academy Press, 
2101 Constitution Ave, NW, Washington, DC 20418, (800) 624-6242. 

Measuring Up is designed to illustrate performance assessment tasks that could be used with 
fourth graders to support reform efforts in mathematics instruction. The book presents 13 
prototype assessments which are meant to be examples of assessment possibilities, not 
examples of ready-to-use assessments that provide an adequate sample of the NCTM 
standards 

Each task description contains the following information; time allotment, student grouping, 
assumed background knowledge of students, task description, student materials, 
rationale/explanation, and "protorubric ” The 13 assessments have the following features: 

( 1 ) they take 1-3 class periods, (2) some require collaboration; (3) most require student 
explanations; (4) they present a series of questions related to a general activity; (5) most have 
a single correct or best response, although a few are more open-ended; and (6) some have 
Spanish translations. 

Scoring is based on “protorubrics, " beginning scoring guides that are not yet fully fleshed out. 
All are task-specific and use a three-point scale — high, medium, or low response. Abstracting 
across "protorubrics," the following types of things are included in the "high” category; 
correctness of response, evidence of conceptual understanding, flexibility of thinking, clarity 
of presenting results, and problem solving (However, the “protorubrics” do not themselves 
identify their own content in this manner, nor do all include everything on this list.) The 
"protorubrics” are not, in themselves, sufficient to interpret results; the “rationale” section for 
each assessment is also necessary 

Tasks were pilot-tested to ensure that they work as intended No other technical information 
is available Many samples of student responses are included 

(TC# 500.3MEAUPP) 



.McDonald, Joseph P., Sidney Smith, Dorothy Turner, et al. Graduation by Exhibition — 
Assessing Genuine Achievement, 1993. Available from: Association for Supervision and 
Curriculum Development, 1250 N. Pitt St., Alexandria, VA 22314, (703) 549-9110, 
fax (703) 549-3891. 

This book describes a strategy for school reform called "planning backwards from 
exhibitions." In this approach, schools define a vision of what they want for graduates by 
proposing a task they want graduates to do well. Having set the vision, they have students 
perform the task and compare the vision against actual performance. Then they plan 
backwards what students would need to know and be able to do at various grades or ages in 
order to be able to do well on the task. 
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This booklet describes this process with three case studies, each proposing a different task 
"platform" against which they guage student success — writing a position paper, inquiring and 
presenting, and participating in discussion seminars 

(TC# 150.6GRAEXA) 



McTighe, Jay. Developing Performance Assessment Tasks: A Resource Guide, October 
1994. Available from: .Maryland Assessment Consortium, c/o Frederick County Public 
Schools, 115 E. Church St., Frederick, MD 21701, (301) 694-1337. 

This is a notebook of performance assessment "must reads." The authors have assembled 
their favorite papers on definitions, overview of performance assessment, and designing 
performance tasks and criteria The notebook also contains Maryland's learner outcomes 

(TC# 150.6DEVPEA) 



.McTighe, Jay. Maryland Assessment Consortium: A Collaborative Approach to 

Performance Assessment, 1991. Available from: Maryland Assessment Consortium, 
c/o Frederick County Public Schools, 115 E. Church St., Frederick, MD 21701, 
(301)694-1337. 

This entry' contains handouts from a presentation by the author in 1991 The following topics 
are covered: 

1 A description of the consortium — what it is and what it does 

2 An overview of the process used for developing performance tasks, and review criteria 
for performance tasks. 

3 Examples of three performance assessment tasks developed by the consortium; one 
math problem-solving task for grade six and two fifth grade reading tasks. .All tasks are 
scored using a four-point holistic scoring guide. Scoring appears to be generalized 
rather than tied to individual tasks. The reading tasks, for example, are scored using the 
same, generalized scoring guide. 

(TC# 500.3MARASC) 



McTighe, Jay. Teaching and Testing in Maryland Today: Education for the 2Ist Century, 
1992. Available from: Maryland Assessment Consortium, c/o Frederick County Public 
Schools, 115 E. Church St., Frederick, MD 21701, (301) 694-1337. 

This 13-minute video is designed to introduce parents and community members to 
performance assessment 

(TC# 150.6TEATEMV) 
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Mead, Nancy. lAEP (International Assessment of Educational Progress) Performance 
Assessment (Science and Math), 1992. Available from: Educational Testing Service, 
Rosedale Rd., Princeton, NJ 08541, (609) 734-1526. 

This document supplements the report by Brian Semple (also described in this bibliography) 
(TC» 600 6PER.ASS) The document contains the administrators manual, scoring guide, 
equipment cards, and released items from the Second International Assessment of Educational 
Progress in science and mathematics 

(TC# 50C.31AEPPA) 



.VIedrich, Elliott A., and Jeanne E. Griffith. International Mathematics and Science 
Assessments: tVhat Have fVe Learned?, 1992. Available from: National Technical 
Information Service, US Department of Commerce, 5285 Port Royal Rd., Springfield, 
VA 22161.(703) 487-4650. 

This report provides a descnption of the international assessments of math and science (First 
International Mathematics and Science Studies, 1960's; Second International Mathematics and 
Science Studies, 1980's; and First International .Assessment of Educational Progress, 1988), 
some of their findings, and issues surrounding the collection and analysis of these data. It also 
offers suggestions about ways in which new data collection procedures could improve the 
quality of the surveys and the utility of hmn e reports 

(TC# 000.6INTMAS) 



Meltzer, L. J. Surveys of Problem-Solving & Educational Skills, 1987. Available from: 
Educator's Publishing Service, Inc., 75 .Moulton St., Cambridge, MA 02138. 

Although this is an individual test published primarily for diagnosing learning disabilities for 
students aged 9-14, it has some interesting ideas that could be more generally applied. There 
are two parts to the test — a more-or-less standard individualized aptitude test, and a series of 
achievement subtests. The math subtest involves a fairly standard test of computation. The 
interesting part comes in the scoring. Each problem is scored on choice of correct operations, 
ability to complete the word problem, efficiency of mental computation, self-monitoring, self- 
correction, attention to operational signs, and attention to detail (one point for evidence of 
each trait). 

After the entire subtest is administered, the teacher is guided through analysis of the student's 
strategies in completing the task — efficiency of approaching tasks, flexibility in applying 
strategies, style of approaching tasks, attention to the task, and responsiveness during 
assessment. (Each area is assigned a maximum of three points for the presence or absence of 
three specific features of performance. For example, under "efficiency" the students get a 
point if he or she does not need frequent repeating of instructions, a second point if the 
student implements the directions rapidly, and a third point if the student perseveres to 
complete the task.) Examples of scoring are included 
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A fair amount of technical information is included This covers typical performance, factor 
analysis, inter-rater reliability, relationship to other measures of performance, and comparison 
of clinical groups. 

(TC# 010.3SUROFP) 



Miller, Edward, Ed. The Push for New Standards Provokes Hope and Fear — and Both Are 
Justified. Located in: The Harvard Education Letter 9. Seotember/October 1993. 

pp. 1-6. 



The author's position is that new standards, in and of themselves, will not reform or improve 
•American education in math We also need to pay equal attention to developing teachers' 
skills and to providing all students with equal opportunity to learn. Roadblocks to real reform 
include; ( 1 ) outmoded tests; (2) lack of teacher sophistication in math; (3) pressure to cover 
all the material in textbooks; (4) lack of time for planning and learning; and (5) lack of 
materials and resources. The article also includes other topics such as the need for standards 
that emphasize different skills and not just attaining higher levels of traditional skills, lessons 
from other countries, and vignettes demonstrating the various points made in the article. 

(TC# 500.5PLSNES) 



.Models of Authentic Assessment Working Group. Superitem Tests in Mathematics, 
undated. Available from: National Center for Research in Mathematical Sciences 
Education, 1025 W. Johnson St., Madison, WI 53706, (608) 263-3605 

The document we received is a set of 20 open-response tasks that are designed to measure the 
statistics, measurement and geometry knowledge of middle school students (grades 7-9). 

Each of the 20 tasks has a set of four questions, each of which has only a single correct 
answer From a sample problem, it appears that all responses are scored right/wrong using 
task-specific scoring guides. 

The materials we received did not contain a description of which specific goals were covered 
by each question, sample student responses, scoring guides, contextual information, nor 
technical information 

(TC# 500.3SUPTM) 



.Mullen, Kenneth l;. Free- Response Mathematics Test, 1992. Available from: American 
College Testing Program, PO Boi 168, Iowa City, LA 52240, (319) 337-1051. 

This was a paper presented at the annual meeting of the National Council on Measurement in 
Education, San Francisco, April 1992 
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This paper reports on a study by ACT that compares multiple-choice, open-response, and 
gridded response item formats on reliability, difficulty and discrimination. In gridded response 
items, students fill in "bubbles" that correspond to the answer rather than choosing the answer 
from a given list "Testlets" were designed to cover the same content and have the same test 
length for each format Results indicated that all formats had about the same reliability, there 
was good rater agreement on the open-ended problems, and grid and open-ended problems 
discriminated better between students with different achievement levels. The correlation 
between performances on the v arious types of items ranged from 0 5 to 0 7 

A few sample problems are provided. All open-response questions used scoring criteria that 
emphasize degree of correctness of the response and were tied to the task (i.e , there was a 
different scoring guide for each problem) 

(TC# 500.3FREREM) 



Mumme, Judy. Portfolio Assessment in Mathematics, 1990. Available from: California 
Mathematics Project, University of California — Santa Barbara, 522 University Rd., 
Santa Barbara, CA 93106, (805) 961-3190. 

This booklet describes what mathematical portfolios are. what might go into such portfolios, 
how items should be selected, the role of student self-reflection, and what might be looked for 
in a portfolio Many student samples are provided Criteria for evaluating portfolios include: 
ev idence of mathematical thinking, quality of activities and investigation, and variety of 
approaches and investigations No technical information is included. 

(TC# S00.6PORASI) 



National Center for Research on Evaluation, Standards, and Student Testing (CRESST). 
Assessing the Whole Child, 1994. Available from: CRESST, Graduate School of 
Education, 145 Moore Hall, 405 Hilgard Ave.. Los Angeles, CA 90024, (310) 206-1532. 

This 17-minute video illustrates and describes one teacher's use of assessment in her 
third/fourth grade classroom. The teacher describes various assessment techniques, why she 
uses each, and what each shows about students. There are clips of how it looks when used in 
the classroom with students. The tape shows use of standardized tests, performance 
assessment, conferences, journals, and projects. The tape also shows criteria for choosing 
materials for maffi portfolios, student, self-reflection, and student self-assessment. The 
teacher's basic philosophy is that the more we know students, the better able we are to help 
each succeed This is a high quality production 

(TC# 150.6ASSWHO and TC# l50.6ASSMHCv) 
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National Council of Teachers of Mathematics. Mathematics Assessment: Alternative 
Approaches, 1992. Available from: The National Council of Teachers of Mathematics, 
Inc., 1906 Association Dr., Reston, VA 22091. 

This 71 -minute videotape and guide are for professional educators who want to try new 
approaches to assessment, as described in the book Mathematics Assessment: Myths, Models, 
Good Questions and Practical Suggestions, by Jean Stenmark (also on this bibliography, 

TC# 500 3MATASM) The video is divided into six segments that each require from 8-15 
minutes of viewing time Individual segments could be used during a sequence of teacher 
education or staff development meetings The segments show challenges that confront 
teachers through the school year as they try new approaches to assessment Classroom clips 
are included Many suggestions are provided for how to follow-up each segment. (A copy of 
the book Mathematics Assessment: Myths, Models, Good Questions and Practical 
Suggestions is included with the video and guide when purchased ) Topics covered include 

• Goals of alternative assessment 

• Characteristics of "authentic assessment" 

• Designing and using good assessment tasks 

• Writing in mathematics 

• Using interviews and checklists 

• Portfolios 

• How to implement these ideas 
(TC# 500.6MATASAV) 



National Science Foundation. Educating Americans for the 21st Century: A Plan of Action 
for Improving Mathematics, Science and Technology Education, 1983. Available from: 
National Science Board Commission on Precollege Education in Mathematics, Science 
and Technology, Forms & Publications Unit, 1800 G St. N>V, Room 232, 

Washington, DC 205S0, (202) 357-3619. 

This is not strictly a document regarding assessment, but rather a statement of what students 
need to know and be able to do in science and math, As such, it also provides an outline for 
what assessments should measure 

(TC# 000.5EDUAMF) 
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Newmann, Fred M., Walter G. Secada, and Gary G. Wehlage. A Guide to Authentic 
Instruction and Assessment: Vision, Standards and Scoring, 1995. Available from: 
Wisconsin Center for Educational Research, School of Education, University of 
Wisconsin, 1025 W. Johnson St., Madison, W I 53706, (608) 263-4200. 

The authors' premise is that innovative teaching techniques (e g , cooperative learning, group 
■•'scussions, hands-on experiments and videos) do not guarantee a change in the "intellectual 
quality" of what 'students are asked to do For example, "a portfolio that shows a variety of 
student work over a semester might replace the final exam taken in one sitting, but the 
portfolio itself could be filled with tasks" devoted to remembering and listing isolated bits of 
intormation Thus, the merit of any teaching technique should be judged by its ability to 
improve the "intellectual quality of student performance " 

The paper is devoted to defining "intellectual quality," developing criteria forjudging the 
intellectual quality of 'he school work students are asked to perform, forjudging the 
intellectual quality of assessment tasks, and for assessing the intellectual quality of student 
work The criteria are designed for use in any content area and thus serve to define delivery 
and performance standards across the curriculum and across teaching methods. Specific 
examples of the criteria are provided in math and social studies Some samples of student 
work are included 

(TC# 050.3GUIAUI) 



NicoII, Richard, Kathy Jacobson, Jim Ryboit. Curricu'um-Based Alternative Assessment of 
Mathematics (CBAAM) IThird Grade and Sixth Grade J A Report to Teachers, April 1993. 
.Available from: Mt. Diablo Unified School District, 1936 Carlotta Dr., Concord, CA 
94519, (510) 682-8000, ext. 4135- 

This review 's based on reports of the 1993 third and sixth grade assessments Students 
respond to three extended problems, some of which have right/wrong answers and some of 
which are more generative For example, one third grade problem requires students to plan a 
city Students are directed to list “buildings and places needed to have a working 
community,” pick 8-10 from the list, place these on a map, and describe the rationale for the 
placement. Students do both group and individual work. 

A generalized, holistic, six-point scoring guide is tailored to individual problems In the 
generalized version, a “6” is: “complete response with a clear, coherent, unambiguous, and 
elegant explanation, includes a clear and simplified diagram when appropriate; communicates 
effectively to the reader; shows understanding of the open-ended problem’s mathematical 
ideas and processes, identifies all the important elements of the problem ” The document 
includes three scored student responses for each problem 
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The document also includes a rationale for alternative assessment, and the context for the Mt. 
Diablo assessment. No technical information is included The author has given educators 
permission to copy this document for their own use 

(TCU 500.3MTDIAC2) 



Nicholls, John G., Paul Cobb, Erna Yackel, et al. Students' Theories About Marhematics 
and Their Mathematical Knowledge: Multiple Dimensions of Assessment. Located in: 
As sessing Higher Order Thinking in Mathematics , Gerald Kulm (Ed.), 1990. Available 
from: American Association for the Advancemerit of Science, 1333 H St. NW, 
Washington, DC 20005, (301) 645-5643. 

This paper reports on a series of studies on student attitudes toward mathematics and their 
relationship to mathematical knowledge and understanding Dimensions of attitudes toward 
math were 

1 How motivated students are to do math 

2 Student beliefs about what causes success in math 

3 Student view s of the benefits of learning math 
All items are included 

(TC# 500.3STITHA) 



North Carolina Department of Public Instruction. Linking Curriculum, Instruction, and 
Assessment: The Patterns, Relationships, and Pre-Algebra Strand, Grades 3-8 
Mathematics, circa 1993. Available from: North Carolina Department of Public 
Instruction, 301 N. Wilmington St., Raleigh, NC 27601, (919) 715-1895. 

In 1989, North Carolina adopted the revised Standard Course of Study for Mathematics K-8 
In 1991, the state began field testing open-ended and objective test items in grades 3-8 to be 
used as part of the end-of-grade tests. The new testing program emphasizes higher-order 
thinking and problem solving. In order to assist teachers to move to a new kind of instruction 
and assessment, this 60-page document was developed It describes five to seven math 
objectives for each grade (3-8) and gives two sample test items (one multiple-choice and one 
open-ended) for each objective Scoring on the open-ended problems uses a three- or four- 
point general rubric that is very sketchy 

No technical information nor samples of student work are included Some instructional ideas 
for each objective are provided 

(TC# 500.3LINCIII) 
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Office of Educational Research and Improvement (OERI). Improving Math and Science 
Assessment. Report on the Secretary 's Third Conference on Mathematics and Science 
Education, 1994 . Available from: U.S. Government Printing Office, Superintendent of 
Documents, Mail Stop: SSOP, Washington, DC 20402 

This 1 5-minute video and companion booklet covers highlights of the Secretary's Third 
( 'on/erence on Mathematics and Science Education: Improving Math and Science 
Assessment during which more than 550 educators, researchers and policymakers gathered to 
talk about such questions ?s Why must assessment change"* What forms of math and scier 
assessment can t elp American students succeed in these subjects"* How can districts 
reforming assessment assure that tests are fair for students of all races and income levels and 
both genders"* How can better assessments fuel the drive toward comprehensive reform of 
,\merican education and higher academic standards"* Recommendations, insights and 
information from the conference are incorporated into the video and the accompanying report 

(TC# 000.6 IMP. MASv & 000.6IMPMASI) 



Oregon Department of Education. Oregon Dimensions of Problem Solving, 1994. Available 
from: Office of Assessment and Technology, f )regon Department of Education, 

255 Capitol St NE, Salem, OR 97310, (503) 378-8004. 

The Oregon Department of Education began giving open-ended math problems to a sample of 
students in grades 3, 5, 8, and 1 1 in 1992 The five short, written problems used in each 
grade in 1992 are included in this document, as are student instructions. Responses are 
scored on four dimensions, or traits ( I ) conceptual understanding of the problem — the ability 
to interpret the problem and select appropriate information to apply a strategy for solution, 

(2) procedural knowledge — the ability to demonstrate appropriate use of math, (3) skills to 
solve the problem, and (4) communication — the ability to use math symbols well and the 
ability to explain the problem solution 

Each trait is scored on a scale of 1-5 The 1994 scoring guides are included in this document 
along with one sample student problem No technical information nor anchor papers are 
included 

(TC# 500.30RDIPS) 
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Oregon New Standards Project. Student Portfolio Handbook— {Quantify/ Science/ 

Mathematics — Field Trial Version/Elementary, 1994. Available from: Oregon Dept, of 
Lducation, Public Service Bldg., 255 Capitol St. NE, Salem, OR 97310, (503) 378-8004. 
Also available from: New Standards at the National Center on Education and the 
Economy, 39 State St., Suite 500, Rochester, NY 14614, (716) 546-7620, 
fax (716) 546-3145. 

This document is the elementary science and mathematics portfolio student handbook 
developed by the Oregon New Standards Project It is organized around student content and 
process goals in Oregon's Cenificate of Initial Mastery (CIM) — number sense, estimation, 
geography, measurement, statistics, patterns, physical, earth, space and life systems skills, 
science as inquiry, problem solving, interpreting results, connections, and communication 

The document includes a description of these areas, examples of items that could be selected 
for the portfolio that demonstrate student ability in these areas, entry cover sheets, and a self- 
review checklist Students are responsible for assembling their own portfolios. The document 
also includes draft scoring guides and a letter of introduction. No technical information or 
sample student work is included 

(TC# 000.3STLPOQ) 



Pa;idey, Tej. Power Items and the Alignment of Curriculum and Assessment. Located in: 
Assessing Higher Order Thinking in Mathematics . Gerald Kulm (Ed.), 1990. Available 
from: American Association for the Advancement of Science, 1333 H St. NW, 
Washington, DC 20005, (301) 645-5643. 

The author presents a philosophy and approach for thinking about the development of a test 
of mathematics problem solving, and provides some examples of multiple-choice and short- 
answer "power" questions developed by the California Assessment Program. 

The author maintains that typical content-by-process matrices used to specify the content of 
tests tend to result in tests that measure minuscule pieces of information that are fragmented 
and non-integrated. The author prefers to have assessment tasks that are broader in focus and 
cut across several process/content areas, so that in order to get the right answer, students 
must use skills like organizing information, representing problems, and using strategies 

(TC# 500.6POWITA) 



Pandey, Tej. A Sampler of Mathematics Assessment, \99\. Available from: California 
Department of Education, Bureau of Publications, Sales Unit, PO Box 944272, 
Sacramento, CA 94244, (916) 445-1260. 

This sampler describes the types of assessment that the California Assessment Program (CAP) 
is proposing to support curricular reforms Illustrated and discussed are open-ended 
problems, enhanced multiple-choice questions, inve.stigatir ns. and portfolios. These four 



NWR£L, September 1995 


51 




Mathematics 


lest Center. (503) 275-9582 




C, t 





types ot activities are intended to measure mathematical understandings that students develop 
over a period of several years 

This monograph includes a definition of "mathematical power" — the ultimate goal of 
mathematics instruction, guidance in the characteristics of assessment tasks that will 
encourage and measure power, a few sample student responses to problems, and help with 
implementation of alternative assessment 

All performance-based techniques will use a six-point holistic scale This scale is briefly 
described The scale will be tailored for individual tasks 

(TC# 500.3SAMMAA) 



Paulson, Leon. Portfolio Guidelines in Primary Xlath^ 1992. Available from: Multnomah 
County Educational Service District, PO Bo* 301039, Portland, OR 97220, 

(503) 255-1842. 

This monograph provides some assistance with getting started with portfolios in the primary 
grades The author believes that the most important purpose for mathematics portfolios is to 
prompt students to take control of their own learning. Therefore, the student should be in 
control of the portfolio (The author, however, also points out that there might be other 
audiences and purposes for the portfolios that might have to be addressed ) 

The author provides some ideas for tasks that students could do to generate material for the 
portfolio, provides some very practical suggestions for getting started, gives ideas for 
activities to encourage student self-reflection, and shows some draft holistic criteria for 
evaluating portfolios 

An example of the user-friendly way this monograph provides practical help is "Remember, 
the portfolio is telling a story. Each item in a portfolio is there for a reason It should not 
require a mind reader to figure out why it is there A portfolio entry includes a piece of work 
plus information that makes its significance clear — the reason it was selected, the learning 
goals illustrated, student self-reflections, and (always!) the date " 

Note: A new version is due out in the fall of 1 994 

(TC# 500.6PORGUP) 



Paulson, Leon, and Pearl Paulson. An Afternoon to Remember: A Portfolio Open House for 
Emotionally Disabled Students, 1992. Available from: Multnomah County Educational 
Service District, PO Box 301039, Portland, OR 97220, (503) 255-1842. 

Reynolds School District adapted Crow Island's "portfolio night" for use with severely 
emotionally disabled students This paper describes how the afternoon was set up, what 
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happened, student debriefing sessions, and changes in format based on student comments. 

(TC# 000.6AFrREP) 



Perlman. Carole. The CPS Performance Assessment Idea Book, November 1994. Available 
from: Chicago Public Schools, 1819 W. Pershing Rd., Chicago, IL 60609. 

This handbook was developed to assist educators to develop performance assessments Its 
most notable feature is a bank of over 90 sets of rubrics for assessing student performance in 
various grade levels and subject areas — reading, writing, mathematics, science, social studies, 
and fine arts There are also well written sections on how to develop (and evaluate the quality 
of) rubrics and performance tasks 

(TC# 000.3CPSPEA) 



Pfeiffer, Sherron. NIM Game Project, 1994. Available from: Southeast EQUALS, 

14 Thornapple Dr., Hendersonville, NC 28739, (704) 692-4078. 

The assessments described in this document are appropriate for upper elementary and middle 
school students. Two project tasks are included, one individual and one group. The projects 
require students to create a game that requires application of math skills. These extended 
projects are used after students have had many opportunities to work with different kinds of 
NIM games Projects become part of a portfolio that shows growth over time. 

The projects are scored using criteria specific to these tasks. The criteria revolve around the 
quality of the game and its usefulness in teaching the math skills specified The project 
instructions and scoring guide are included .Also included is a manual with (a) lots of help on 
how to use NIM games with students, (b) 10 HIM games developed by others, and (c) a 
general discussion of the characteristics of worthwhile instructional tasks No sample student 
work nor technical information is included 

(TC# 500.3NLMGAP2) 



Porter, Andrew C. Standard Setting and the Reform of High School Mathematics and 
Science, 1995. Available from: Wisconsin Center for Education Research, School of 
Education, Univenity of Wisconsin-Madison, 1025 W. Johnson St., Madison, WI 
53706, (608) 263-4200. 

The author reports on a study that looks at the effects of increased enrollment in academic 
classes resulting from raising course-taking graduation requirements. Previous studies 
discovered that increased graduation requirements did not raise dropout rates and that, 
indeed, students were taking more academic classes, especially science and math. This study 
examined whether the affected academic courses were "watered down" to accommodate 
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weaker and less motivated students. The author found that courses were not watered down. 
The conclusion is that standards for high school students have, indeed, been raised 

(TC# 000.6STASER) 



Pritchard, Diane. Student Portfolios— Are They Worth the Trouble?, 1992. Available from: 
Sisters Middle School, PO Box 555, Sisters, OR 97759, (503) 549-8521. 

This paper was written by a middle school math and English teacher It provides practical 
help with how to set up a portfolio system in math by describing her purpose for having a 
portfolio, the types of activities included, and activ ities to get students to self-reflect 
(including an idea for tests) 

(TC# 500.3STUPOT) 



Psychological Corporation, The. Alternative Model Assessment Package (AMAP ) — 
Mathematics, 1993. Available from: The Psychological Corporation, Order Service 
Center, PO Box 839954, San Antonio, TX 78283, (800) 228-0752, fax (512) 270-0327. 

The AMAP package consists of mathematics and language arts performance tasks for students 
in grades 3*10 The math tasks consist of a series of questions on a common theme. For 
example, in the level 5 task, students use ratios to determine dimensions for flags. Each task 
takes two l*hour class periods Students work individually Student responses to the 
investigation are scored as a whole — the various parts and steps do not receive separate 
scores. A six-point holistic rubric is used — only one score is given The rubric emphasizes 
completeness, communication, relevance, logic, and computation. 

This document includes one sample task plus administration instructions and a scoring guide 
No sample student work nor technical information is included 

(TC# 500.3AMAP-M) 



Psychological Corporation, The. GOALS: A Performance-Based Measure of Achievement, 
1992. Available from: The Psychological Corporation, Order Service Center, 

PO Box 839954, San Antonio, TX 78283, (800) 228-0752. 

GOALS ;s a series of open-response questions that can be used alone or in conjunction with 
the MAT-7 or SAT-8, or any achievement test. Three forms are available for 1 1 levels of the 
test covering grades 1-12 in the subject areas of science, math, social studies, language and 
reading Each test (except language) has ten items. The manual states that the math 
questions assess student problem solving, communication, reasoning, connections to other 
subjects, estimation, numeration, geometry, patterns, statistics, probability and algebra. Tasks 
are multiple, short problems The manual draws the distinction between the approach taken in 
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GOALS (efficiency in large-scale assessment), and the related publication "Integrated 
Assessment System" which has fewer tasks pursued in more depth. 

Responses are scored on a scale of 0-3, where 0 is "response is incorrect" and 3 is "accurate 
and complete with supporting information " The scoring guide is generalized and is used for 
all problems Scoring can be done locally or by the publisher. i here is good assistance with 
scoring philosophy and procedures There are two sample student performances for each 
score point for each question 

The holistic scales are combined in various ways to provide indicators of overall conceptual 
understanding and various specific aspects of problem solving and using procedures. These 
are. however, not scored directly. Rather, it is analogous to multiple-choice tests in which the 
correct items are combined in various ways to give subtest scores 

Both norm-referenced (percentiles) and criterion-referenced (how students perform on 
specific concepts) score reports are available A full line of report types (individual, summary, 
etc.) are available 

The materials we obtained did not furnish any technical information about the test itself 
(TC#510.3GOALS) 



Psychological Corporation, The. Integrated Assessment System: Mathematical Performance 
Assessment, 1991. Available from: The Psychological Corporation, Order Service 
Center, PO Box 8399S4, San Antonio, TX 78283, (800) 228-0752. 

This is a series of 14 tasks designed to be used with students in grades 2-8. Two task 
booklets were designed for each grade level, but can also be used in spring testing of the 
grade below or fall testing of the grade above Each task booklet presents a problem situation 
that is expanded on and applied to a series of questions. For example, various task booklets 
focus on symmetry, breaking a tie in an election, planning an orchard to maximize yield, and 
bar codes. Questions involve sue' things as figuring out an answer and explaining how the 
solution was reached, and generating a principle and applying it to a new situation. 

Solutions are scored either holistically (0-6) or analytically (four. 4-point scales). The 
performance criteria represent generalized features of problem solving and so can be used to 
score performance on any task. The holistic scale is used to provide an overall picture of 
performance; raters look for quality of work, evidence of understanding of concepts, logical 
reasoning, and correct computations. The analytical traits are: reasoning, conceptual 
knowledge, communication, and procedures. Scoring can be done either locally or by the 
publisher 

The set of materials we obtained includes a brief description of the scoring rubrics and one 
example of a scored student test. Technical information was not included 

(TC# 500.3INTASM) 
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Qin, Zhining. Minnesota Mathematics State Assessment, 1992-93. Available from: OfTice 
of Teaching and Learning, Minnesota Department of Education, Capitol Square, 

550 Cedar St, St. Paul, MN 55101, (612) 296-6104, fax (612) 296-3348. 

The Minnesota Mathematics State Assessment has both open-ended and multiple-choice 
questions for grades 5. 8. and 1 1 designed to assess skills such as: problem solving, 
communication in math, applying math to real-life problems, explaining thinking, and 
estimation The purpose of the assessment is to provide information to make a judgment 
about the effectiveness of the mathematics program at both the local and state level. 

This document contains sample open-ended materials from the 1992 and 1993 assessments. 
.At each grade level students are asked a series of questions which require them to solve a 
problem, draw a picture, explain their thinking, write a problem, etc Scoring is task specific 
Sample student work is included 

Substantial technical studies have been conducted including validity, rater agreement, and 
inferential and descriptive statistics Materials are publicly available at cost. 

(TC# 500.3MNMATG) 



Regional Educational Laboratory Network Program. Improving Science and Mathematics 
Education — A Database and Catalog of Alternative Assessments, 1994. Available from: 
Document Reproduction, Northwest Regional Educational Laboratory, 101 SW Main 
St., Suite 500, Portland, OR 97204, (503) 275-9519, fax (503) 275-9489. 

The intent of this database/catalog is to provide information to those who want to learn more 
about current efforts to redesign assessment to match changing goals for students and ideas 
about instruction. The database contains descriptive information on alternative assessments in 
science and mathematics that cover all grade levels, levels of assessment (classroom to 
international), and purposes. Developers include teachers, school districts, state departments 
of education, national governments, universities, and other research and development 
institutions. The most appropriate users are assessment specialists, curriculum coordinators, 
and others responsible for developing alternative assessments. 

Note that the database contains only information about the assessments. Actual copies of the 
assessment instruments themselves are available from the contact people listed in the catalog. 
The database operates using FileMaker Pro software from Claris. 

(TC# 000.1DATCAA2) 
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Regional Educational Laboratory Network Program. Improving Science and Mathematics 
Education — A Toolkit for Professional Developers: Alternative Assessment, 1994. 
Available from: Document Reproduction, Northwest Regional Educational 
Laboratory, 101 SW Main St., Suite 500, Portland, OR 97204, (503) 275-9519, 
fax (503) 275-9489. 

The 552-page Toolkit was designed cooperatively by all 10 regional labs as a teacher 
professional development resource It is a compilation of activities and supportive materials 
that serve both as an alternative assessment resource and a means of engaging teachers in 
dialogue about changing the way mathematics and science have traditionally been taught and 
assessed The Toolkit contains: 

• Information and professional development activities on the topics of rationale for 
alternative assessment, integrating assessment and instruction, design options for 
alternative assessment, being a critical consumer of assessments, grading/reporting, and 
planning effective professional development 

• Twenty-one sample assessments 

• Bibliography of resources 

(TC# 000.6TOOKIP2) 



Riverside Publishing Company, The. Arizona Student Assessment Plan — Mathematics. 
Available from: The Riverside Publishing Co., 8420 Bryn Mawr Ave., Chicago, IL 
60631,(800) 323-9540. 



Arizona is developing open-response, on-demand assessments in math, reading, writing, social 
studies, and science for grades 3, 8, and 12. The math test has been used for about two years; 
the science test is still under development. The mathematics test requires no manipulatives or 
equipment. Students provide short answers to a series of questions surrounding a common 
theme, such as a rock climbing competition or a “pizza spinner.” They also sometimes 
provide explanations for responses or show computations. 

Scoring is task-specific. Points are assigned for the correctness of the response or the number 
of responses given. 

The tests were developed by Riverside and are under copyright restriction until at least 1996 
Permission to use any part of the assessment must be granted both by Riverside and the 
Arizona Department of Education 
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Riverside Publishing Company, The. California Program for Learning Assessment — 
Mathematics Petformance Assessments, 1994. Available from: The Riverside 
Publishing Company, 8420 Bryn Mawr Ave., Chicago, IL 60631, (800) 323-9540. 

The California Program for Learning Assessment addresses language arts and mathematics in 
eight levels tor grades 3-10 The math tests are designed to be given in 45 minutes and have 
two parts Part 1 contains two open-ended math problems which take approximately 30 
minutes to complete Part 2 has seven multiple-step, multiple-choice problems which take 
approximately 1 5 minutes to complete The test is designed to assess problem solving, 
application of knowledge, and communication skills rather than knowledge of specific facts or 
operations The open-ended problems require written responses and are completed 
individually Both open-ended and multiple-choice questions are thought-provoking. 

Open-ended responses are scored using a 0-4 point, generalized, holistic rubric where “4” 
"Shows a complete understanding of the problem and addresses all relevant mathematical 
ideas. Exhibits sound reasoning and draws logical conclusions. Communicates clearly 
through the use of appropriate charts, graphs, diagrams, illustrations, and/or words. Provides 
computation (where required) adequate for the solution of the problem.” Although somewhat 
sketchy, this oibric attempts to address the "big” outcom.es in the NCTM standards To help 
the scorer, the general rubric is tailored to each particular problem. 

The materials we received mention a pilot test in 1 7 California schools, but no details are 
given 

(TC# 500.3CALPRL) IN-HOUSE USE ONLY 

Riverside Publishing Company, The. Performance assessments for ITBS, TAPandITED 
(various levels and subject areasf 1993. Available from: The Riverside Publishing 
Company, 8420 Bryn Mawr Ave., Chicago, IL 60631, (800) 323-9540. 

Riverside is publishing a series of open-response items in the areas of social studies, science, 
mathematics, and language arts, Nine levels are available for grades 1-12. They supplement 
achievement test batteries available from the publisher: ITBS, TAP, and ITED. Each level 
uses a scenario to generate a series of related questions, some of which have only one right 
answer, and others of which are more open-ended and generative. Tests take 1 ’/2 to 2 hours 
depending on grade level. 

No information about scoring, no sample student performances, and no technical information 
v/ere included in the materia's we received. However, the publisher’s catalog indicates that 
scoring materials are available and that the tests are normed, 

(TC# 060.3PERAST) IN-HOUSE USE ONLY 
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Romberg, Thomas A. Assessing Mathematics Competence and Achievement, 1989. 
Available from: National Center for Research in Mathematical Sciences Education, 
Wisconsin Center for Educational Research, University of Wisconsin, School of 
Education, 1025 W. Johnson St., Madison, W1 53706, (608) 263-4200. 

This paper describes the author's view of what it means to be literate mathematically It then 
describes the instructional and assessment implications of this goal. The author believes that 
we need to assess not only mathematical knowledge but also the structure of the knowledge 

(TC# 500.5ASSMAC) 



Romberg, Thomas A. The Domain Knowledge Strategy for Mathematical Assessment, 1987. 
Available from: National Center for Research in Mathematical Sciences Education, 
Wisconsin Center for Educational Research, School of Education, 1025 W. Johnson St., 
Madison, W1 53706, (608) 263-4200. 

This document provides a brief overview of the "Domain Knowledge" strategy used by the 
National Center for Research in Mathematical Sciences Education to assess math knowledge 
of students This approach is contrasted to the typically used "Content by Behavior Matrix" 
approach in which content topics are crossed with behavior (usually some form of Bloom's 
taxonomy) The author maintains that this approach is outdated; the behavior dimension fails 
to reflect contemporary notions of how information is processed and the content dimension is 
an inadequate way to describe what is meant by "knowing mathematics" 

The "Domain Knowledge" approach involves making a "map" or network of a concept 
domain. This reflects a more integrated and coherent picture about knowledge. These maps 
can be used to generate tasks, assessment criteria, and formats that get at both "correctness" 
of responses and the strategies used to arrive at the answer. 

(TC# 500.6DOMKNS) 



Romberg, Thomas A. Evaluation: A Coat of Many Colors, Available from: National 
Center for Research in Mathematical Sciences Education, Wisconsin Center for 
Educational Research, University of Wisconsin, School of Education, 1025 W. Johnson 
St., Madison, WI 53706, (608) 263-4200. Also located in: Mathematics Assessment and 
Evaluation: Imperatives for Mathematics Educatora . Thomas A. Romberg (Ed.), 1992. 
Available from: State University of New York Press, State University Plaza, Albany, 
NY 12246. 

This paper describes the impact of assessment information on decision making and describes 
the ways in which assessment must change if it is to have a positive impact on such decisions 

(TC# 500.6EVACOM) 
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Romberg, Thomas A. Mathematics Assessment and Evaluation: Imperatives for 

Mathematics Educators, 1992. Available from: State University of New York Press, 
State University Plaza, Albany, NY 12246. 

This book covers several interesting topics with respect to assessment in math Specifically 

1 How tests communicate what is valued 

2 How current tests will not promote the recommendations in the NCTM standards 

j Various considerations when developing tests: calculators, how to adequately model 
knowledgeable students, etc 

4 Setting up assessment that is intended to influence instruction 

Although authoritative, this book is written in a very academic style, which makes it less 
accessible to general readers Articles that are most relevant to this bibliography are entered 
separately 

(TC# 500.6MATASE) 



Romberg, Thomas A., and Linda D. Wilson. Alignment of Tests with the Standards. 

Located in: Arithmetic Teacher . September 1992, pp. 18-22. 

The authors make the argument that teachers teach to tests Therefore, if we want the NCTM 
standards to be implemented we need to have tests that reflect the standards. The authors 
feel that many current norm-referenced tests do not match the standards. Finally, they present 
tasks from several innovative assessments that they feel do reflect the standards. 

(TC# 500.6ALITEW) 



Romberg, Thomas A., Linda D. Wilson, 'Mamphono Khaketia, et al. Curriculum and Test 
Alignment. Located in: Mathematics Assessment and Evaluation: Imperatives for 
Mathematic s Educators . Thomas A. Romberg (Ed.), 1992. Available from: State 
University of New York Press, State University Plaza, Albany, NY 12246. 

This article reports on two studies on the alignment of current standardized tests and 
alternative assessments to the NCTM standards. Results showed that current standardized 
tests are weak in five of six content and process areas, and place too much emphasis on 
procedures and not enough on concepts The authors present several examples of test 
questions that they feel do match the standards 

(TC# 500.6CURTEA) 
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Romberg, Thomas A., £• Anne Zarinnia, and Steven R. Williams. The Influence of 
Mandated Testing on Mathematics Instruction: Grade 8 Teachers' Perceptions^ 1989. 
Available from: National Center for Research in Mathematical Sciences Education, 
Wisconsin Center for Educational Research, School of Education, 1025 W. Johnson St., 
Madison, WI 53706, (608) 263-4200. 

This monograph reports on the first of a sequence of studies on mandated testing in 
mathematics This study was a large-scale questionnaire survey to find out from Grade 8 
teachers how influential mandated testing was on their teaching of mathematics The results 
of the study showed that nearly 70 percent of the teachers reported that their students take a 
mandated test Secondly, because teachers know the form and character of the tests their 
students take, most teachers make changes in their teaching to reflect this knowledge Third, 
the kinds of changes teachers make are in contrast to the recommendations made by the 
NCTM standards Specific examples are given 

Although this paper does not describe an alternative assessment device, it does provide 
reasons for seeking alternative ways of assessing math 

(TC# 500.6INFMAT) 



Schoenfeld, Alan H. Teaching Mathematical Thinking and Problem Solving. Located in: 
Toward the Thinking Curriculum: Current Cognitive Research . Loren B. Resnick & 
Leopold E. Klopfer (Eds.), 1989. Available from: Association for Supervision and 
Curriculum Development, 1250 N. Pitt St., Alexandria, VA 22314-1403, (703) 549-9110. 

Although this article is more about defining what mathematical problem solving is than about 
assessment, it presents an interesting visual way to represent how students spend their time 
when solving a problem It also compares a plot of time use for a good problem solver to a 
plot for an inefficient problem solver 

Essentially, the plotting procedure involves tracking the sequence in which people use 
different steps in the problem-solving process (reading the problem, analyzing the problem, 
exploring a solution strategy, planning, implementing a strategy, and verifying the results) and 
the amount of time spent on each, Good problem solvers spend a lot of time analyzing and 
planning, with many self-checks on "how it is going." Poor problem solvers tend to fixate on 
a possible line of attack and pursue it relentlessly even when it is clearly not going well 
Additionally, there are very few stops to self-check on how it is going 

(TC# 500.5STOWTET) 
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Scottish Examination Board. Standard Grade - Revised Arrangements in Mathematics, 

1987. Available from: Mr. K. Hutchon, Examination Officer, Ironmills Rd., Dalkeith, 
Midlothian, Edinburgh, Scotland, EH22 ILE, UK (031) 663-6601. 

The Scottish Examination Board prepares end-of-course tests for a variety of high school 
subjects to certify’ level of student competence The course syllabus for mathematics calls for 
cov erage of number, money, measurement, geometry, trigonometry, algebra, and 
graphs/tables The goals of the course are: knowledge/understanding, reasoning/applications, 
and investigating. There are two main parts of the assessment in math — written tests 
(developed by the Examination Board) and performance assessments (conducted by teachers 
according to specifications developed by the Examination Board). The two parts are 
combined to rate student competence on a scale of 1-7 ( 1 being high), both overall and for 
each goal 

On-demand written tests, developed each year, cover knowledge/understanding and 
reasoning/applications. Three levels of the test are available Foundation, General and Credit 
Depending on the percent correct score, students can obtain ratings of 1 or 2 on the Credit 
level. 3 or 4 on the General level, and 5 or 6 on the Foundation level. All questions are short 
answer or multiple-choice and are scored for degree of correctness of the answer 

The hands-on performance assessments must cover a range of activities including the 
identification and use of real data, the use of measuring or drawing instruments, the 
recognition or exploration of a pattern, conjecture, or proof; and the formulation of a 
mathematical model Candidates w rite up their investigations in the form of a report 
Performances are scored on “understanding and organizing the task,” “carrying out the task.” 
and "communication ' A total of 12 marks (points) is available General criteria for level 
designation are provided as well as the point conversions described above 

The package of materials we received included the course syllabus, specifications for the 
written and performance assessments, and copies of the written tests for 1993 It did not 
include technical information nor sample student responses 

(TC# 500.3.MATSTG) 



Secada, Walter G., Sherian Foster, and Lisa Byrd Adajian. Intellectual Content of 

Reformed Oassrooms. Located in: NCRMSE Research Review 4. Winter 1995, pp. 3-8. 
Available from: National Center for Research in Mathematical Sciences Education, 
Wisconsin Center for Education Research, University of Wisconsin, 1025 W. Johnson 
St., Madison, WI 53706, (608) 263-7582, fax (608) 263-3406. 

.As part of a study on reform in mathematics education, the authors are developing a 
classroom observation form containing indicators of the classroom's "intellectual substance " 
There are 10 scales intended to describe a lesson's content in terms of teacher and student 
behavior, student engagement, and the shared norms of the class revealed through the 
interaction patterns of the class The 10 scales are 
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• Mathematical Concepts — the content of the lesson 

• Use of Mathematical .Analysis — student engagement in analytical math beyond rote uses 
of algorithms, eg, searching for math patterns, making math conjectures, or justify ing 
conjectures 

• Depth of Knowledge and Student Understanding — the depth of student know ledge 
being developed in a lesson 

• Mathematical Connections 

• Cross-Disciplinary Connections — 

• Value Beyond the Class 

• Mathematical Discourse and Communication — the degree to which talking is used to 
understand math 

• Locus of Mathematical .Authority —the extent to which authority is shared with students 

• Social Support for Student .Achievement — conveying high expectations 

• Student Engagement in Doing Mathematics — student motivation 

The article includes a description of the scales and a sample classroom example but no detail 
on scoring and no technical information 

(TC# 500.4INTCOR) 



Semple, Brian. Assessment of Achievement Programme — Mathematics: Second Survey 
1988. Available from: Brian Semple, Principal Research Officer, New St. Andrews 
House, Room 4/51a, Edinburgh, Scotland EHl 3SY, UK (031) 244-4388. 

The ‘ .Assessment of Achievement Programme (AAP)’’ was established by the Scottish Office 
Education Department in 1981 to monitor the performance of pupils in grades 4, 7, and 9 
The 1989 report “Mathematics: Second Survey 1988” reports on the 1988 mathematics 
assessment. The assessment covered geometry, algebra, estimation, and statistics (depending 
on grade level) 

Assessment tasks used two formats written and practical However, the report we have does 
not describe these two formats, nor explain how responses were scored Schools in the 
assessment sample were also invited to comment on their mathematics program 
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The document we have includes the rationale for the assessment and description of student 
performance. No technical information is included 

(TC# 500.3ASSACP) 



Semple. Brian McLean. Performance Assessment: An International Experiment, 1991. 
Available from: Educational Testing Service, The Scottish Office, Education 
Department, Rosedale Rd.. Princeton. NJ 08541, (609) 734-5686. 

Eight math and eight science tasks were given to a sample of thirteen-year-olds in five 
volunteer countries (Canada. England. Scotland. USSR, and Taiwan) This sample was 
drawn from the larger group inv olved in the main assessment The purpose of the assessment 
was to provide an information base to participating countries to use as they saw fit. and to 
examine the use ot performance assessments in the context of international studies 

The 16 hands-on tasks are arranged in two 8-station circuits Students spend about five 
minutes at each station performing a short task Most tasks are "atomistic" in nature, they 
measure one small skill For example, the 8 math tasks concentrate on measuring length, 
angles, and area, laying out a template on a piece of paper to maximize the number of shapes 
obtained, producing given figures from triangular cut-outs, etc Some tasks require students 
to provide an explanation of what they did .AJI 16 tasks are included in this document, 
although some instructions are abbreviated and some diagrams are reduced in size the 
complete tasks, administration and scoring guides are available from ETS 

Most scoring is right/wrong; student explanations are summarized by descriptive categories 
There is also observation of the products of students' work 

Student summary statistics on each task are included T here is a brief summary of teacher 
reactions, student reactions, the relationship between student performance on various tasks, 
and the relationship between performance on the multiple-choice and performance portions of 
the test A few sample student performances are included 



(For related information, .see iWancy Mead, al.so li.sted m this hihtiography.) 

(TC# 600.3PERASS) 



Serrano, Claudia. A Look at Portfolio Assessment in San Diego High School's Sophomore 
House, 1991. Available from: San Diego City Schools, 4100 Normal St., Room 3133, 
San Diego, CA 92103, (619) 298-8120. 

This paper describes an interdisciplinary (physics, math, and English) portfolio system for 
tenth graders that supports block scheduling in an inner city magnet school. Students keep a 
notebook of all work in each class Class portfolios are developed from selected work in the 
notebook Class portfolios are used as the basis for the culminating ‘House Portfolio” in 
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which students select work to demonstrate that they have attained specified learning goals. 
The “House Portfolio” also includes written reflection and a final exhibition of mastery 

The document includes student instructions for assembling the portfolio, an entire student 
portfolio, instructions for a formative oral presentation of their portfolio, checklists and 
evaluation forms, and assistance with reflective writings and exit exhibitions No technical 
information is included 

(TC# 000.3LOOPOA) 



Shoecraft, Paul. MOVE IT Math Concepts I, \9SS-1992. Available from: Mathematics 
Education Initiative, University of Houston-Victoria, 2506 E. Red River, Victoria, 

TX 77901,(512) 576-3151. 

The MOVE IT Math ( 'oncepts I and MOVE IT Math Levels of Use Questionnaire together 
form a two-pronged assessment strategy to promote the implementation of Le' el 1 of the 
MOVE IT math program MOVE IT math is a K-6, university-supported professional 
development program that advocates mathematics instniction based on the use of 
manipulatives to address a wide variety of learning styles (e g., visual, auditory, kinesthetic). 
It consists of three half-hour inservices (Level 1 is one of the three). 

The Math Concepts I assessment instrument is a constructed response, paper-and-pencil test 
of the concepts of ( 1 ) equals as balanced, (2) exchanging “up” in base ten (3) being an 
exchanging expert, and (4) exchanging “down” in base ten. A measurement item involving 
adding feet and inches is included to assess ability to take into account the context in which 
numbers appear 

Copies of the Concepts 1 test, answers, and test results from six years of use are available 
They may be used as it or modified, with appropriate citation to thf; authors 

(TC# 500.3MOV1TM) 



Silver, Edward A., and Jeremy KilpatricL Testing Mathematical Problem Solving. 

Located in: The Teachina and Assessing of Mathematical P *oblem Solving . Randall 
Charles and Edward Silver (Eds.), 1988. Available from: National Council of Teachers 
of Mathematics, Inc., 1906 Association Dr., Reston, VA 22091. 

This paper discusses two topics: how assessment can inform instructional decision making 
and how it communicates what we value. The authors propose that xhe National Assessment 
of Educational Progress and many other math tests do not provide the type of information 
needed for the improvement of mathematics instruction. The information useful for 
improvement of instruction would be types of errors kids make, how automatic mathematical 
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processes are, and the cognitive structures and abilities associated with expertise in the 
domain being tested. 

(TC# 500.6TESMAP) 



Stecher, Brian M. Describing Secondary Curriculum in Mathematics and Science: Current 
Status and Future Indicators^ 1992. Available from: RAND, 1700 Main St, 

PO Box 2138, Santa Monica, CA 90407. 

The author describes what could go into an indicator system of the health of science and 
mathematics education He concludes that current data sources for these indicators are 
inadequate 

(TC# 000.6DESSEC) 



Stenmark, Jean Kerr. Mathematics Assessment: Myths, Models, Good Questions, and 
Practical Suggestions, 1991. Available from: National Council of Teachers of 
Mathematics, 1906 Association Dr., Reston, VA 22091. 

This monograph was designed for teachers in the elementary grades. It is a collection of 
examples of assessment techniques that focus on student thinldng. Topics include the 
rationale for new ways of assessing mathematics, the necessity of integrating assessment and 
instruction, designing performance assessments (most emphasis is on designing the task, 
although sample holistic and analytical trait scoring systems are shown), what to look for 
during classroom observations and interactions (including questions to ask to get at various 
types of thinking), portfolios (including types of items to include and the types of information 
they can demonstrate about students, and criteria for evaluation), student self-assessment, and 
hints to make assessment work in the classroom. No technical information is provided. 

(TC# 500.3MATASM) 



Stuart Foundations Project Teachers. Language Arts and Science Performance Assessment 
Sandler, 1993. Available from: San Diego City Schools, 4100 Normal St., Room 3133, 
San Diego, CA 92103, (619) 298-8120. 

This document includes 1 2 performance assessment project tasks developed by teachers for 
students in grades K-3 and 4-8. Some of the tasks are interdisciplinary (science, math, 
writing.) and some are subject specific (mostly writing). The tasks themselves are interesting 
and may provide ideas for others developing performance assessments. This document does 
not include rationale or context for the tasks, technical information, or sample student work 
The performance criteria listed for each project are sketchy 

(TC# 100.3LANARS) 
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Surber, John R. Mapping as a Testing and Diagnostic Device^ 1984. Located in: Spatial 
Learning Strategies — Techniques. Applications, and Related Issues. C. D. Holley & D. 

F. Dansereau (Eds.). Available From: Academic Press, 1250 6th Ave., San Diego, 

CA 92101. 

The book is a general discussion of the advantages of. and procedures for, integrating the 
production of cognitive networks into instruction The premise is that knowledge of facts, 
rules, algorithms, etc. is only part of what students need to know They also need to know 
how these facts fit together to form a body of knowledge Without knowledge of the 
interrelationships, students are not likely to remember the facts or be able to use them 
correctly when they are remembered 

The Surber paper discusses a particular type of cognitive networking scheme — mapping — and 
its use in assessment of knowledge structures The basic procedure consists of taking a 
completed map for the topic to be tested, and deleting portions in various ways. Students 
then complete the map given various types of cues 

(TC# 000.6MAPASA) 



Surber, John R., Philip L. Smith, Frederika Harper. MAP Tests, 1981 - undated. Available 
From: John R. Surber, University oF Wisconsin-Miiwaukee, Department oF Educational 
Psychology, Milwaukee, WI 53201, (414) 229-1122. 

Our review is based on four reports from the author: Testing for Misunderstanding (John R 
.Surber and Philip L Smith. Educational Psychologist . 1981. 16. .3, pp. 165-174; Technical 
Report No. I, Structural Maps of Text as a Learning Assessment Technique: Progress 
Report for Phase /; Surber. Smith, and Frederika Harper, undated. University of Wisconsin- 
Miiwaukee; Technical Report No. 6, The Relationship Between Map Tests and Multiple 
Choice Tests, Surber, Smith and Harper. 1982. University of Wisconsin-Miiwaukee; and 
Mapping as a Testing and Diagnostic Device, Surber: Spatial Learning Strategies . 1984. 
Academic Press, Inc . pp 213-233 (also available as TC# 000 6MAPASA) 

These reports and papers describe the development of map tests as an assessment technique to 
identify conceptual misunderstandings that occur when students learn from text. The purpose 
is to diagnose student understanding in order to plan instruction In this testing technique, the 
test developer graphically represents concepts and their interrelationships in a map. Then, 
information from the map is systematically removed. Students complete the map shells Four 
different levels of deletion associated with different types of content clues are described 
Maps are scored by comparing the student-completed version to the original. Scoring 
involves looking both at the content included or omitted from the map and the proper 
relationship between this content Report #6 describes scoring in more detail 

The authors did a series of studies on this technique, reported on in "Mapping as a Testing 
and Diagnostic Device " They found good interrater reliability and good consistency between 
developers of "master maps " They report on comparisons to multiple-choice tests 
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Text maps and tests can be constructed in any content area at any grade level. The specific 
examples in these materials come from chemistry (matter), study skills, and sociology (the 
development of early warfare) 

■\ manual, designed to teach students how to construct concept maps, is included in 
Report #1 The authors have given educators permission to copy these documents for their 
own use 

(TC# 150.6MAPTES) 



Thompson, Linda. Portfolio Model, \994~95. .Available from: Linda Thompson, Chief 
.Moses Junior High School, 2215 Westshore Dr., Moses Lake, WA 98837, 

(509) 766-2661. 

The author is experimenting with the use of portfolios in her seventh grade math classes 
Students are asked to included in their portfolio at least one entry for each of mathematics as 
problem solving, mathematics as communication, mathematics as reasoning, mathematics 
connections, estimation, number sense and numeration, concepts of whole number operations, 
whole number computation, geometry and spatial sense, measurement, statistics/probability, 
fractions and decimals, patterns and functions A single entry might serve to illustrate more 
than one skill Thus, the portfolio is designed to align with the NCTM standards. 

The packet of materials includes the direction sheets for students, a sheet for each portfolio 
entry (revised in 1995 because students needed additional help selecting entries and reflecting 
on them), a self7peer rating sheet for group cooperation, a list of required content for a parent 
conference portfolio, the student version of a scoring guide for individual portfolio entries, 
and tw'o activities designed to assist students to better understand and use the criteria (added 
in 1995) The scoring guide is holistic and uses a four-point scale where “4” is: “This 
response provides proof that you really understand the mathematical concepts you are 
demonstrating. You have communicated your understanding so well there is no question that 
you have mastered the ideas being explored.” Thus, students are scored on conceptual 
understanding and communication; the packet contains no rubrics for problem solving, 
reasoning or connections 

The packet also does not include technical information nor sample student work The author 
has given permission for educators to copy the materials for their own use 

(TC# 500.3PORMOD) 
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Tittle, Carol Kehr and Deborah Hecht. Mathematics Assessment Questionnaire: A Survey 
of Thoughts and Feelings for Students in Grades 7-9 — Technical Report and Manual for 
Users,, 1990. Available from: Graduate School and University Center, City University 
of New York, 33 W. 42nd St., New York, NY 10036, (212) 642-2262. 

The purpose of the Mathematics Assessment Questionnaire is to survey student thoughts and 
feelings about learning mathematics in classroom activity settings and to provide information 
that complements assessments of mathematical understandings. The questionnaire has 143 
statemei\ts asking students about their awareness of self-regulatory thinking and skills, 
affective beliefs (interest, value, confidence, anxiety), and motivational and attributional 
beliefs These are assessed in the context of three instructional activity settings: during class, 
working with other students, and homework 

The Questionnaire is available as part of a computer-based, classroom assessment package 
The package includes student and teacher computer-based forms and a teacher guide. Both 
the teacher guide and the teacher computer program include instructional suggestions linked 
to each area of assessment 

(TC# 500.3MATASQ) 



Vermont Department of Education. Vermont Mathematics Portfolio Project: Grade Eight 
Benchmarks, 1991. Available from: Vermont Department of Education, Vermont 
Mathematics Portfolio Project, 120 State St., Montpelier, VT 05602, (802) 828-3135. 

This document provides lots of samples of grade eight student work that illustrate different 
scores for each of the seven analytical traits used in the Vermont Mathematics Portfolio 
Project Samples were taken from the 1991 portfolio pilot 

(TC# 500.3GRAEIB) 



Vermont Department of Education. Vermont Mathematics Portfolio Project: Grade Four 
Benchmarks, 1991. Available from: Vermont Department of F iucation, Vermont 
Mathematics Portfolio Project, 120 State St., Montpelier, VT 05602, (802) 828-3135. 

This documents provides lots of samples of grade four student work that illustrate different 
scores for each of the seven analytical traits used in the Vermont Mathematics Portfolio 
Project Samples were taken from the 1991 portfolio pilot. 

(TC# 500.3GRAFOB) 
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Vermont Department of Education. Vermont Mathematics Portfolio Project: Resource 
fiooA, 1991. Available from: Vermont Department of Education, Vermont 
Mathematics Portfolio Project, 120 State St., Montpelier, VT 05602, (802) 828-3135. 

This document includes sample performance tasks taken from portfolio entries submitted by 
teachers as part ot Vermont's 1991 math portfolio pilot project, a resource bibliography, and a 
list of suggested readings The purpose is to provide colleagues with tasks that have worked 
well with students to promote problem solving This is meant as a companion document to 
the Teacher's Guide (TC= S00.3TEAGVI). 

(TC# 500.3RESBOO) 



V ermont Department of Education. Vermont Mathematics Portfolio Project: Teacher's 
Guide, 1991. .Vvailable from: Vermont Department of Education, Vermont 
Mathematics Portfolio Project, 120 State St., Montpelier, VT 05602, (802) 828-3135. 

This document presents V'^ermont's current view of what should go into a mathematics 
portfolio, provides detailed information about the scoring criteria for portfolio entries and the 
portfolio as a whole, discusses how to develop tasks that will invite student problem solving, 
and provides help with how to manage the portfolios This is a companion piece to the 

Resource Book (TC= 500.6RESB()0). 

( TC# 500.3TEAGLT) 



Webb, Noreen. Alternative Strategies for Measuring Higher Order Thinking Skills in 

Mathematics: The Role of Symbol Systems, 1991. Available from: CRESST, University 
of California— Los Angeles, 145 Moore Hall, Los Angeles, CA 90024. (213) 825-471 1. 

This document presents an overview of a study that is currently taking place at CRESST in 
which students are asked to represent problems in various equivalent ways (graphs, tables, 
equations, word problems, and diagrams). The premise is that if a student really understands 
a problem, he or she should be able to solve the problem presented in any format, and 
translate from one format to another. Examples are provided of problems represented in 
different ways 

(TC# 500.6 ALTSTF) 
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Webb, Noreen. Collaborative Group Versus Individual Assessment in Mathematics: 

Processes and Outcomes - CSE Technical Report 352, March 1993. Available from: 
Center for Research on Evaluation, Standards, and Student Testing (CRESST), 
University of California, Los Angeles, CA 90024, (310) 206-1532. 

The author reports on a study designed to determine the extent to which information from 
group tasks can be used to make inferences about the abilities of individual students, and also 
determine what additional information about student skills is provided from group tasks 

The study used 53 seventh grade students who worked in small groups to calculate the costs 
of long distance phone calls Two weeks later students worked individually on a similar 
problem. Work was scored on nine dimensions, eg, “recognizes that call has multiple 
minutes." and “applies single cost to each additional minute." The nature of individual student 
behavior in the group was also analyzed, e g . students ( 1 ) solved the problems correctly aloud 
with little or no assistance from others, and (2) expressed difficulty with the problems (made 
errors or asked questions indicating that they were confused) 

Results showed that; 1 1 ) students working in groups showed uniformly high levels of 
performance; (2) some lower-achieving students did better when assessed individually than 
would be predicted from pretest scores, but only in cases when they used the resources of the 
group to understand the procedures; (3) however, in general, there was a large discrepancy 
between individual and group performance, (4) analyzing the group process skills of individual 
students shed considerable light on individual student understanding and on their collaborative 
skills, (5) students learn more from the group experience if they have been taught how to 
work productively in teams 

{TCU 500.6COLGRV) 



Webb, Norman, and Thomas A. Romberg. Implications of the NCTM Standards for 
Mathematics Assessment. Located in Mathematics Assessment and Evaluation; 
Imperatives for Mathematics Educators. Thomas A. Romberg (Ed.), 1992. Available 
from: State University of New York Press, State University Plaza, Albany, NY 12246. 

This paper provides a good summary of the NCTM standards, both goals for students and 
standards for assessment It uses four of the standards for assessment to develop criteria for 
assessments: 

1 The assessment instrument should provide information that will contribute to decisions 
for the i.mprovement of instruction. 

2 The assessment instruments should be aligned with the instructional goals, the goals for 
the overall program, and a holistic conceptualization of mathematical knowledge 

3 The assessment instruments should provide information on what a students knows 
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4 The results from one assessment instrument should be such that when combined with 
results from other forms of assessment, a global description is obtained of what 
mathematics a person or group knows. 

The authors then illustrate their points with several assessment tasks that they feel would elicit 
the correct behavior from students (These generally have only one correct answer and appear 
to be scored for degree of correctness ) 

(TC# 500.6IMPNCM) 



Webb, Norman L. and .Arthur F. Coxford, Eds. Assessment in the Mathematics Classroom, 
1993 Yearbook, 1993. Available from: The National Council of Teachers of 
Mathematics, Inc., 1906 Association Dr., Reston, V.A 22091. 

This book contains a number of papers that discuss classroom assessment in grades K-12 
Specific topics include student self-assessment, assessing problem solving, EQUALS, NCTM 
standards, and integrating assessment and instruction. 

(TC# 500.6ASSMAC) 



Whetton, Chris. Key Stage 1, 1993, Teacher’s Pack, \993. Available from: HMSO 
Publications Centre, PO Box 276, London SW8 5DT, England, UK. 

This document contains all administration materials for the 1993 assessment. The assessments 
consist of a combination of hands-on and paper-and-pencil activities for primary students 
English, science and mathematics are covered In science and math, some activities are scored 
for the correctness of the answer and some are scored for correctness of approach or 
explanation For example, one math task consisted of adding and subtracting using a small 
number of objects (The student must get three out of four correct to be scored as "pass.") 
One science task has students draw pictures or verbally explain what forces are acting on a 
raft as it floats on the water. (Responses are scored correct if the student conveys the 
knowledge that there are forces acting down and up on the raft.) Scoring is always tied 
directly to the task, and tasks usually are designed to cover discrete skills or pieces of 
knowledge. 

All tasks are administered by the classroom teacher in large and small group settings Results 
of the 1 993 administration are not yet available, so it is unknown how long the most current 
version takes (The 1993 assessment was greatly streamlined from the 1992 assessment which 
took 24 hours, including English ) 

(TC# 070.3KEY193) 
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Wilkins, Cynthia. Student Portfolio (Algebra), cira 1994, Available from: Northwest 
Rankin Attendance Center, 62 Terrapin Hill Rd. North, Brandon, MS 39042, 

(601) 825-2522. 

This algebra or pre-algebra portfolio is intended to represent all that a student has learned 
during the year The short document describes each entry (e g., math autobiography, concept 
explanation, journal topic, specific skills, etc ) and emphasizes student self-reflection 
Although point values for each entry are suggested, there are no criteria for assigning points 
It has been used for seventh and eighth graders taking algebra Student directions are 
available 

(TC# 530.3STUPOA) 



W ilson, Linda D. A Theoretical Frameyvork Linking Beliefs with Assessment Practices in 
School Mathematics, April 1994. Available from: University of Delaware, College of 
Education, Educational Development, Newark, DE 19716, (302) 831-2573, 
fax: (302)831-4110. 

This paper describes a "pure constructivist" notion of teaching, assessment practices that align 
with pure constructivism, and how consistent the NCTM assessment standards are with a 
notion of pure constructivism The paper concludes that most current ideas of assessment in 
math, including the NCTM assessment standards, represent only "soft" constructivism and do 
not adhere to the strictest tenets of a constructivist notion of how students learn 

(TC# 500.6THEFRL) 



Wilson, Mark. Measuring Levels of Mathematical Understanding. Located in: 

Mathematics Assessment and Evaluation: Imperatives for Mathematics Educators . 
Thomas A. Romberg (Ed.), 1992. Available from: State University of New York Press, 
State Universit'f' Plaza, Albany, NY 12246. 

The premise of this article is that if we want students to be reasoners and thinkers, we need to 
move from tests that fragment knowledge into "atomistic" pieces, each of which are assessed 
independently of the others, to assessment procedures that reveal student understanding of the 
concepts in a domain and their interrelationships Many current tests are based on lists of 
skills, each of which is tested separately. "The primary focus of a mathematics testing 
methodology based on an active, constructive view of learning is on revealing how individual 
students view and think about key concepts in a subject. Rather than comparing students' 
responses with a 'correct' answer to a question so that each response can be scored right or 
wrong, the emphasis is on understanding the variety of responses that studt nts make to a 
question and inferring from those responses students' levels of conceptual understanding." 

The author presents a few examples One is the SOLO taxonomy which looks at degree of 
formal reasoning (See the ( 'oUis-Romherg K'" S00.3COLROM on this bibliography.) 




7,1 



NWTl£L, September 1995 
Test Center, (503) 275-9582 



* % 



Mathematics 



This is a very technical and theoretical article and points up the need to be well grounded in 
current theory before beginning to develop math assessments. 

(TC# 500.6MEALEM) 

Zarinnia, E. Anne, and Thomas A. Romberg. .4 Framework for the California Assessment 
Program to Report Students' Achievement in Mathematics. Located in: Mathematics 
Assessment and Evaluation: Imperatives for Mathematics Educators. Thomas A. 
Romberg (Ed.), 1992. Available from: State University of New York Press, State 
University Plaza, Albany, NY 12246. 

This paper takes the position that assessment affects instruction, and therefore, regardless of 
the other purposes for the assessment, the instructional implications of our assessments must 
be taken into account "If one acknowledges student learning as the central mission of 
schooling, it further suggests that not only the tasks, but also the system and structures for 
gathering accountability information and reporting the data, should be designed with 
instructional needs in mind " 

Other points made by this paper are 

1 VVe need to change the view of math held by many teachers and the general public, that 
math is a set of rules and formalisms invented by experts that everyone else is to 
memorize The authors maintain that both the test itself and the way results are reported 
will influence these perceptions 

2 Mathematical power means that citizens can use math to solve day-to-day problems. 

This means we need to seek evidence of students using, reflecting on. and inventing 
mathematics in the context of value and policy judgments. These experiences should be 
built into our instruction and assessments 

Implications for turning power over to studeiits are also discussed 
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Mathematics Bibliography 
Index Codes 



A — Type 

1 = Example 

2 = Theory how to assess/rationale for 

alternative assessment 

3 = Content/what should be assessed 

4 = Related: general assessment, 

program evaluation, results of 
studies, technology, attitudes 

B — Purpose for the Assessment 

1 = Large scale 

2 = Classroom 

3 = Research 

C — Grade Levels 

1 = PreK-K 

2 = 1-3 

3 = 4-6 

4 = 7-9 

5 = 10-12 

6 = Adult 

7 = Special education 

8 = All 

9 = Other 

D — Content Covered 

1 = General math 

2 = Algebra 

3 = Geometry/trigonometry 

4 = Statistics/probability 

5 = Precalculus/calculus 

6 = Other 

7 = All/Any 
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E — Type of Tasks 

1 = Enhanced multiple choice 

2 = Constructed response short 

answers 

3 = Long response/essay 

4 = On-demand 

5 = Project 

6 = Portfolio 

7 = Group 

8 = Other than written 

9 = Cognitive map 

F — Skills Assessed 

1 = Knowledge/conceptual 

understanding 

2 ” Application of concepts 

3 = Persuasion 

4 = Critical thinking/problem solving, 

reasoning/decision making 

5 = Group process skills 

6 = Quality of writing/communication 

7 = Student self-reflection 

8 = Process 

9 = Comprehension 
10= Attitudes 

G — Type of Scoring 

1 = Task specific 

2 = General 

3 = Holistic 

4 = Analytical Trait 



5 Mathematics 



O 



DOE=Dcparuncni of Education or 
State Department of Education 
A I Alberta Ed. (TC«500 3 DIamaP) 

A I Alberta Ed ac>^500 3DiPEXP) 

AlAlginadC^ 150 6;EM292) 

A1 Aipcr :Tc^ 500 n.tKiMP) 

A I Appalachia Ed Lab < rc \ w i \sm* 

A [.Aurora Pub Schools i ic ^ sum.\?) 
.Al.AZStud .Assin Prog Tc « " UKisiM2i 
.A I Badger 30 M H 0 M 1 

.AlBa.xter dc-- 5oo bM.\rPK.\) 

AIBCMinistn of Ed dc- )ou mkp! r \ i 
AIBellingham Pub Sch . tc -muo ihki \n si 
.AlC.AASSm Coll (Tc= 15i*f>CH ACf)l > 

AICA Assm Prog > rc as amm 
AICA DOEi lc ' ' otjsn sis» 

.A 1C. A DOE l it ^<00 \Q[ I S()» 

AlCharles dc-5o(.» hMou ioh > 

A 1 Clark dc =500 nu ipma» 

\lClark'TC=5oo.A\iMmM) 

AlClarridge (Tc= i5o oimpnh ) 

AIColliSiTC = 500 KOI ROM > 

.AlCrowIcN rR'::5u() .^sn .map* 

AlCTDOE ( TC’'»00 .KONCOC! 

AlCTB McGraw-Hill > k 

AlCTB McGraw-Hill ( re =500 K'l'MMAi I 
AlCurnculum Corp iTe-500 niATCi Pi 
.AlDoig I rC= 500 lACTASMi 
AlEd Testing Service < it -^oo on ai pmar» 
,AlEd Testing Service i rc -oon kiisai a> 
.AlFinston dc=5o<Msn vsi > 

AIFt Ha>sEd De\ Center dc '00 isi a\sM) 
A I Hall (10=500 3 ALBGRN) 

A I Hartman ( tc=5oo 3M.atp( k) i 
A 1 Hibbard (Tc=ooo hKHisrcA 
A I Hibbard ( TC= 500 ^sm sp> 

AlHom (TC =000 M ANCOP* 

A I H> nes ( rc = 500 t k- 5 m a}» i 
AIK atims (TC= 500 .^packi.ti 
A lKnight (TC = 530 3HOWR S) 

AlKulm «TC» 500 6MATASW ) 

A IKY DOE (TC*060 3WRIS94) 

A I Lambdin (TC«500 3 PLapoa) 

A I Lane (TC*5 oo 3 QUacoa) 

AILarter (TC«ioo 6BENCHM) 

.A I Lawrence itc»ooo sino tca i 
A I Lawrence < rc=5oo .ut \c(k > 

A I Leach (TC=5 oo 3.ai tfof ) 

Al Lehman (Tc=5oo .^pkr.vsM) 

A I Lesh (Tc*500 6ASS ai P) 

AIMath Science Ed Bd (TC «5 oo .3Mf vi pn 
AIMD Assm Consort rre- ioo imarasu 
AIMD Assm Consort iTC'^ooo ipfr vsTo 
AIMD DOE (TC» 500 3MSPAPM) 



AIMD DOE (TCaOOO 3COMST) 

A I Mead (TC=500 3IAEPPA) 

AIMeltzer (Tc«oio 3srROFP) 

A I Models of Auth Assm iTC=5U(j 3si ptfmi 
AIN at’ICtr for Res onE%al dc=i5o hASSWnciM 
AINCDept ofPub Instr dc= *00 31 i\cn. 

A I New mann TC= o5(j 301 i.xi 1 > 

AINiColl '11-500 3MTDI.AC:) 

A I OR DOEdc=5uo 30 RDIPS 1 

AlOR New Stand Proj iTC= uoo 3sn pogi 

AlPaulson <Tc=5ou bPORcrP) 

A I Perlman dc»ooo 3 CPSpe ai 
A I Pfeiffer dc=5oo 3 MMgaP) 

.AlPntchard fTC=5oo 3 snpor» 

.AlPsxCh Corp dCs 500 3 V.M.\P*Mi 
A lPsxch Corp rrc=5oo 31 NT.vsm^ 

AlPsxch Corp dC=5lO 3G0A1.S. 

AlQin I rcs 5oo .vmnm.atG) 

AIReg Ed Lab Net Prog dC=oooe>T(X3Klp:! 

A I Riverside Pub Co. (TC»060 3PER.ast) 

A I Riverside Pub. Co (TC«5oo .3 C.xlprij 
A I Semple (TC=6oo 3 PERAss) 

A I Serrano (TC= 000 3L(X)P0A) 

A I Shoecraft iTC=5oo 3Mo\ itsd 
A I Stuart Fndn Proj Teachers i Tc=iou 3L.\n ars) 

A I Thompson (TC»5 oo 3PORMod) 

A ITittle (TC= 500 3 M.atasq) 

A I VA DOE (TC»500 3GRAH)bi 
A I VA DOE (TC=500 3RESB(J0) 

AIVA DOE (TC = 500 3TEAGH) 

AI\T DOE iTC=5oo 3 gr.u:ib> 

A 1 Webb (TC = 500 6.Al.rSTF) 

A I Webb (TC=500 6ASSMAC) 

AlWhetton iTCso-^o 3kF.vi93) 

A I W ilkins ( rc= 530 3s*n poa) 

A2Bagley (Tc= 5()0 6 .asssti)( 

A2CA Assm. Coll dc= I50 6CMAcoi ) 
A2ClarkiTC85oo 3MCTPMai 
A2Comm on Stds for Sch Math i rc«5oo 5ci r.ane) 
A2Conley (TC« i50 6 PRobaa) 

A2Harvey (TCs5oo6Mattec) 

AZKulm (TC« 500 6MAT.ASW) 

A2Kulm (TC=500 6ASSHIO) 

A2Lesh (Tc*300 6 assact) 

A2Lesh (TC*500 scombaa) 

A2McTighe (TC«i5o steatemv) 

A2Mlller (TC«500 5PLSNES) 

A2Nat’l Coun Tchrs Math dc* 50 oomat.\sas) 
A2NC Dept, of Pub Instr. (Tr* 500 3 i.rNcn) 
A2Nicoll (TC»300 3MTDIAC2) 

A20ERI (Tc« 000 6 IMPMASVI) 

A2Pandey dc«5oo isammaA) 

A2Pandcy (Tc*5oo 6 Powitaj 
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AiPerlman iTCa uoo 3CPSPt.\j 

A2Reg. Ed. Lab. Net. Prog. (TC«uoobr(X)MP:i 

A2Romberg (TC»5oo 6/U.ITEW) 

A2Rombcrg (TC»5oo 6D0MKNS) 

A2Romberg (TC«5oo 6 MAT.asE) 
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.A4Fit/.painck iTCd?u bPERPRE» 
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A4Kulm ( re « ^00 6M at asv^ i 
A 4Kulni ( rca?(K)6 Assmo) 



.A4Lesh (Tcsfoob.ASSAe p) 
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A4Romberg (Te'a^oo bcrRTF.V) 
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BICA Assm Prog (Tca^oo ^s-amm.v\ 2) 
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BICA DOE iTCfl5oo jaqi eso) 
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BlClark (TC«5oo 3 aimhim) 
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BiCTB McGraw-Hill (Tc <»060 le vr 
BICTB McGraw-Hill (Te«5oo ictbm.vd 
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BlEd. Testing Service (TC»500 6Naepm.aR) 
BlEd. Tesung Service (TCf»ooo 3MiSALA) 

BIFi Hays Ed. Dev. Center (TC» 5 oo 3st vvsm) 
B I Hall (TC*500 3ALBtlRN) 

B I Hartman (Tc« ?oo 3 Matpot)) 

BIHyncs (Tc»?oo 3K.5 maP) 

BIKV DOE . R = 3KIR1S‘)4> 

BILancr icsi'M nHp\cHMi 
BILawrencc t^\\r\ tc\ < 

BILaurence (TC^?oo 3iT.\coc> 

B I Lesh (TC8500 6ASS ai P) 

BIMadaus iTC3 000 6BR]h.v\i 

BIMD Assm Consort iTC^ ooo 3 mar asl ^ 

BIMD DOE (TCs <00 uisp apm» 

BIMD DOE (TCsOOO JCOMSTi 

B I Mead I re 3 <00 3iAiPPAi 

BIMell/er (TCsoio tsi ROFPi 

BIModcIs of Auth Assm. nc^^ou 3si riiMi 

B I Neu mann <tc3 05o luriAU) 

BlNicoll (TC 8<00 3MTDIAC2) 

BIOR DOE (TC^SK) 30RDIPS) 

B I OR New Stand Proj (TC^Oni) 3s n P( 

B I Perlman (Tc^ ooo 3cpspfa) 
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B2Burton fTC#500 3ADDSEK) 
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B2Greenwood itc<koo 60 N'\ atti 

NWIIFL. September 
Test Center. ( 503 ) 275-9582 



B2Hibbard (TC»ooo 6TOGSTO 
B2Hibbard (TC»50 o 3SEU sP) 

B2Hom (TCsooo vlancop) 

B2Katims dcs <oo 3PACktT» 
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C2Bums (TCi500 SMATLIT) 
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C2CTB McGraw-Hill (TC«500 3CTBMAT) 
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C2Nat’l Ctr for Res onE\al vsswhcivi 

C2NC Dept of Pub Insir ( rc = ?ud 11 i\ct n 

C2NlCOll ( rc=500 3MTDIAC:» 
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C3Baxter (TC» ?oo 6 Matpea> 

C3Bellingham Pub Sch , (Tc= 000 ibelpi si 
C3Burton(TC»?oo saddsek) 

C3CA Assm Prog. (TCs?oo 3 s.amm.aa:i 
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CSAlpcr {TQtt 500 3\ ARIMP) 

CSAppalachia Ed. Lab (Tc= 600 3 .altasm> 
C8BC Ministrv of Ed (TC»ooo 3Bcpfr \> 
C8CA Assm. Coll. (Tc» i50 6CH.ACor^ 
C8Charlcs (Tc»5oo6HOWtofa 
C8CTB McGrau-Hill (Tc »060 3c vr- 5 a) 
C8CumcuIum Corp (Tc»50O3M.\TcrF) 
C8Hibbard (TCaOoO 6T(K}STCi 
CSHihbarci (Tc=500 jsf( i SPi 
CSLesh ( rc'3500 6 ass u P) 

CSMiller <Tc='oo 5PL snesi 
CSP erlman iT( » ooo 3 cpspea) 

C8Ps>ch. Corp (TC»500 3INTASM) 

C8Ps>ch. Corp (TC»?io 30oals» 

C8Reg Ed. Lab Net. Prog. (TC5()00 6T(X3kip2i 
C8Riverside Pub Co (Tc=060 3 Per.\sti 
C8Riverside Pub Co (Tca?oo 3 C.ai.prli 
C8Secada iTc» ?oo aintcoRi 
C8VA DOE iTC=5oo 3resb()Oi 
C8VA DOE (TC=500 3TE.AC}ri) 

C8Webb (Tc»5oo 6 .\ssm.ao 

C9Katims (Tc« 5oo 3P\( ket) 

D I Alberta Ed. (TC»500.30Iam.aP) 

D 1 Alper (To» soo jvarimp) 

DIAZ Stud. Assm. Prog.(Tc» ?oo 3 ,\ristm 2 i 
D 1 Ba.xter (Tc» soo smatpea) 

D I Bellingham Pub. Sch.. (Tc»ooo 3bei pt si 
D I Bums (TC»?00 5MATLm 
DICA Assm Prog. (TC»50o 3s.a.mm.\.u> 
DlClark (Tc»50o 3aimhim) 

DlClairidge (TC» i50 6Impneei 
D IColliS (TC»500 3COLROM) 

D ICT DOE (Tc«ooo 3C0NC0C) 

DICTB McGraw-Hill (TC« 060 3 Cat<»i 
DICTB McGraw-Hill (Tc»5oo tcTBMATt 
DICumculum Corp (TC»500 3M.\TctT>) 



D 1 Doig (TC» 500 3ACT.ASM) 

DlEd. Testing Service (TC»500 6N.aepmaR) 

D 1 Hibbard (TC»ooo 6 T 0 GSTC ) 

D I Hibbard (TC»5 oo 3SEU SP) 

DIKY DOE (TC»060 3K1RIS94) 

D I Lawrence (Tcaooo 6iNTiTCv) 

D 1 Lawrence <tc»5oo jit.acoc) 

DIMD .Assm. Consort (Tc»ooo3M.\r.\sci 
D I Models of Auth Assm. (TC»5oo 3 SITtemi 
D lNat'l Ctr for Res on Eval. (Tc»150 6.asswhciv 
DINC Dept, of Pub. Instr (TC» 5oo3HNa ii 
D l Paulson (TCa ?00 6PORGlT) 

DlPs>ch Corp (Tea 500 3a.M.AP M) 

DlPs>Ch Corp. (TCa500 3INT.\SM) 

DlPs>ch. Corp (TC»5l0 3GOALSi 
DlQin (TCa 500 IStNM.ATG) 

DlReg. Ed. Lab. Net. Prog. (Tcaooo 6TooKJP2) 

D I Riverside Pub. Co. (TCa5oo 3c alprd 
D I Semple (Tcasoo tperass) 

DIShoecraft (Tcasoo 3Movmn 
DIThompSOn (TCaSOO 3POR.MOO) 

D 1 Webb (TCaSOO scolgr V) 

Dl Whetton (TCa070 3KEY193) 

D2Alberta Ed (TCa5oo 3D1PE.\P) 

D2ColliS (TCaSOO 3COLROM) 

D2CTB McGraw-Hill (TCasoo sctbmad 
D 2Curriculum Corp. (Tcasoo smatclt) 

D2D0ig (Tea S00.3ACTASM) 

D2Ed. Testing Service (TcasoooN.AEPMARi 
D2Knight acaS30 SHowas) 

D2LawTence (TcaooooiNTiTCv) 

D2LawTence (TCasoo jitacoc) 

D2Lehman (TCasoo sperasM) 

D2Math. Science Ed Board iTcasoosME.vi pp) 
D2Psvch Corp (TCa 500 j.amap-M) 

D2Psvch. Corp (Tc»soo sintasM) 

D2Ps>ch. Corp (TCaSlO.SCOALS) 

D2Wilkins aca S3o ssttpoa) 

D3Alberta Ed. (TCaSoo sdiamap) 

D3Alberta Ed. (Tcasoo 3 Dipe,xP) 

D3 AZ Stud. Assm. Prog.(TCa soo 3ARJstm2) 
D3CTB McGraw-Hill (TC»soo sctbmat) 

D3Doig (TC» soo 3ACT.VSM) 

D3Lawrence (Tcaooo 6t?m,TCv) 

D3Lawrence (Tc«soo sitacoc) 

D3Models of Auth Assm (Tcasoo ssi ptem) 
D3Ps>ch. Corp (Tc* soo 3amap-M) 

D3Ps>ch Corp. (Tcasoo 3INT ASM) 

D3Psych Corp. (Tcasio jooals) 

D3Thompson ( rc»soo spormod) 

n • Aurora Public Schools (Tc»ooo sscimaP) 
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D4AZ Stud. Assm. Prog(TC» 5oo3.arjstm2) 
D4Baxter (TC» 5oo.6matpea) 

D4CTB McGraw-Hill (Tc«500 3 ctbmaT) 
D4Cumculum Corp (TC«5oo 3 M.atci Pi 
D4Doig (TC» 500 3.\CTASM> 

D4Ed Testing Service (TC»500 eN.Ai- PM.\R> 
D4Kaiims . TC= «oo 3Packeti 
D4KY DOE lie- )6u 3kIRlS04i 
D4Lawrence iTc=uo(j o!sn tca i 
D 4Lavv rence ( TC =500 3t t accx' i 
D 4Models of Auth Assm (Tc=5i)0 vsi ptkmi 
D4Ps>ch. Corp.iTC» 5oo 3 .vm.\P-mi 
D4Ps>ch Corp. .Tc=5oo mnt.asmi 
D4Ps>Ch Corp |TC»MI} 3(K).Al.Si 
D4Thompson iTC=5oo .ipormodi 

D5Lohman ( .'PPR.vsmi 
05Ps>ch Corp i rc= 5oo 3amap-Mi 

D6Alberta Ed (Tc»5oo 3 DIPE\Pi 
D6Bagley <TC» 5oo s.assstd) 

D6Curriculum Corp (TC»500 3MArci Pi 
D6Ps>ch Corp (TC»5oo 3 INTasmi 
D6Tittle (TC= 500 .im.at.vsui 

D'BC Ministry of Ed <TC=ooo ihcpfra) 

□’’CA doe (Tc^iioossn sTSi 
D‘'Ft Hays Ed Dev Cen ;r nc - 5oo is r wsM) 
D'’Kulm(TC= 500 6M.AT.V A i 
DTLcsh(TC^500 6.ASS.A' 

D70R New Stand Proj i rc» uoo .isn pou> 
D7Perlman (Tc»ooo .icpspf.v 
D7Pfei£fer (TC»5oo 3Mmo.aP) 

D7Pritchard (TC»5oo 35X1 poti 

D7Reg. Ed. Lab Net Prog rrc= ooo Mookip:i 

D7Secada (TC» 5oo aintcor) 

D7Seirano (TC» ooo n.ooPOAi 
D7VA DOE (TC»500 3RESB001 
D7VA DOE (TC = 500 ITE.AUIT) 

D7\Vebb (TC»«oo s.altstfi 
D7Webb (TC»500 sassm ac ) 

F ICA DOE (TC» OOO.SSTl’STS) 

ElNat'l Cti. for Res. on Eval (TC>n5o f - vsswhcim 
ElPandey fTc»500 3SAMMAA) 

EIRiverside Pub Co (Tc »500 .kalpri.) 
ElWhetton (TC*oto 3kf.vi93) 

E2Alberta Ed (TC»5ro idipexP' 

E2AZ Stud Assm. Prog.(TC» 5«o varistm:i 
E2Baxter (Tc» 500 smatpea) 

E2Burton (TCk 500 5.ADDSFki 
E2CA DOE (Tf» 000 6511 STS I 
E2Clark (TC»5iX3 3 ,\jmhim) 



E2CT DOE (TC»ooo 3CONCOC) 

E2CTB McGraw-Hill (TC» 060 3Cat-5») 

E2CTB McGraw-Hill (TC»5oo 3CTBMAT) 

E2Doig (TC» 500 3.ACT.ASM1 

E2Ed Testing Service (TC»5oo 6S ae:pm ari 

E2Kulm iTC» 500 6M.at.asw ) 

E2Lane (TCs5oo 3 QL acoai 
E 2Larter (TC»ioo 6BENchm) 

E2Lawxence (TCsooo 6 INTITCv) 

E2Lawtence (TC»5oo 3LT.ac(X'i 
E2M’uaus (Tc» ooo sbriexa) 

E2MD Assm. Consort, (TC» ooo 3 M.ar.\sci 
E2MD Assm Consort. (TC» ooo 3 PEr.asT) 

E2MD DOE (TCa 500 3MSP.\"Ml 
E2Mead (TC»5oo 3IAEPP.A) 

E2Meltzer (TC»oio 3SfROFP) 

E2Models of Auth. Assm. (TC»5oo 3st. ptemi 
E2NiC0ll (TC«500 3MTDIAC2) 

E2Pandey (TC»5oo 3 SAMMaa) 

E2Psych. Corp (TC» 500 3amap-M) 

E2Psych. Corp, (TC»50o 3INTASM) 

E2Psych, Corp. (TC» 51 o 300alsi 
E 2Qin (TC» 500 3ktNMATO) 

E2Reg, Ed. Lab. Net Prog. (TC» 0 j 0 Rrcx)kiP 2 > 
E2Riverside Pub Co (Tc»060 3PrR.AST) 
E2Riverside Pub Co (Tc»500 3C.alprl> 
E2Semple (TC»ojo 3PERass) 

E2Shoecraft (TC» 5 oo jmovttm) 

E2Webb (TC»5oo saltstf) 

E2Whetton (TC»o7o 3kEVi93) 

E3,'\urora Pub Schools (TC »ooo 3scim ap) 

E3AZ Stud, Assm, Prog.(TC» 500 3AR1STM2) 
E3Bums (TC»500 smatut) 

E3Burton (TC»5oo 5.ADDSEk) 

E3CA Assm. Prog (TC»500 3S.vmm,va2) 

E3CA DOE (TCa ooo 6STfSTSl 
E3CA DOE (TC»500 AAgfESO) 

E3CT DOE (TC»ooo 3CONCOC) 

E3CTB McGraw-Hill (TC» o6o 3Cat-5d 
E3Ed Testing Service (TC«500 6NaepM/AR) 
E3Ed, Testing Service <TC« ooo 3MISala) 
E3Finston (Tc»500 3 STI'aso 
E3Ft Hays Ed. Dev, Center (TC« 500 3st,aasm) 
E3Hall (TCK500 3ALBGRN) 

E3Hibbard (Tc»ooo6TOOStc) 

E3Hibbard (Tc»5oo 3SEUsp) 

E3Hom (TOKIOO 3LANC0P) 

E3Hynes (TC«500 3k-5MAP> 

E3K.atims (TC* 500 3 Packet) 

E3Kulm (TC* 500 smatasw) 

E3KY DOE (TC* 060 3KIR1S94) 

E3Leach (TC»5oo 3/ALTFoE) 

E3Lehman (TC*5oo 3PER/\sm) 
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E3Math. Science Ed. Beard (TC«500 3 MEaipp) 

E3MD Assm. Consort. (TC«ooo.3MAiLASC) 

E3MD Assm Consort. (Tc» ooo 3 PER.ast^ 

E3MD DOE (TC» 500 jmsp.apm) 

E3Nal’l Clr for Res. on Eval (TCm^u^asswmcim 
E3NiC0ll (TC=»500 3MTDIAC2) 

E.^OR DOE .TC = 5(JO ^DRDIPSi 
E3Ps>ch Corp < ic- < iu 3 u! \pm> 

E30in <TC» 500 .uiwi vifn 

E3Reg. Ed Lab Net Prog re- o')OftT(x)kiP:i 

E 3 Riverside Pub Co < tcsom; 3PeR.vsT) 

E3Riverside Pub Co ^TC-5oo 3c.u pri.> 

E3Webb (TCs5uo 6Colgrv) 

E4 Alberta Ed (TC 5500 3 niPP.\Pi 
E4Alper (Tc= 5oo 3\ \rimp) 

E4 Aurora Pub Schools i icsooo 3 sc im.um 
E 4A2 Stud Assm Progncs 5o() 3 aristm2) 

E4Ba\ter rics 5ou 6 M.atpeai 
E 4Burton (TCs5oo 5addseKi 
E 4CA Assm Prog (TC85 oo 3sa.mm vv 2 ) 

E4CA DOE (TCs oou t>sn STS) 

E4Clark (Tc»5oo 3.\imhim< 

E4CT DOE <TC 3000 3coscoci 
E4CTB McGraw-Hill (TC^oeo k \r*5a) 

E4CTB McGraw-Hill rTC3500 3 CTBm.\Ti 
E4Ed Testing Service (TC3500 6Na£P\!. jg 
E4Ed, Testing Serv ice i rc s 000 3MIS U.A) 

E4Fl. Hays Ed. De\ Center <tc» 500 3St.vvsm) 

E4Hall (TCs 500 3 .ALBGRN) 

E4Hibbard (TC'aOOO 6TfXiSTa 
E4Hynes (TC»?o() ^k.'Mapi 
E 4Kulm (TCa <00 6 MATASW 1 
E4Lane |TC »500 3gi acoai 
E 4Lawrencc (TCaooo eiNTi rcv) 

E4Lawrence (TC»J()o 3iTACfx:) 

E4Math Sci Ed Board (TCajooJMKAi PPi 
E4MD Assm Consort (TC»ooo 3MARaso 
E4MD Assm Consort. (Tc»ooo 3PER.ast) 

E4Mcad (TC»J 00 3IAEPPA) 

E4Nat'l Coun. of Tchrs. of Math. (TC» 5 oo6Mat asav) 
E4Nat'l Cti for Res. on Eval (TC«iso 6ASSWHavi 
E4NC Dept, of Pub. Instr. (Tc» 5 oo3mnci n 
E4NiCOll (TCk500.3MTDU.C2) 

E40R DOE (TCK500 30RDIPS) 

E4PsVCh Corp (TC«500 3D.T,'JiMt 
E4Psvch Corp (TC«5io 30OA1.S) 

E4Qin (TC« 500 3NtNMATO) 

E4Reg. Ed. Lab Net Prog i rc» ooo 6 T(y)kiP 2 ) 
E4Riverside Pub Co. (Tc»?'/o 3 CUPRi.) 

E4Shoecraft (TCksoo 3mp'.itm) 

E4Webb (TCk5o.3c.coi.or\-) 

E.^BumS (TC«?00 5MATt.'T) 



ESBunon (TCK500 5ADDSEK) 

E5CT DOE (TC»ooo 3CONCOC) 

E5Ed Testing Service (Tc» ()oo 3 Mis.ai.ai 
ESFinston (TC»5oo 3sn .\sn 
E5Hibbard (TC»5oo 3seli sfi 
E5Hom (Tc»ooo 31 ..ancoP) 

E5Katims (Tc» 5oo jp.acket) 

E5KuJm fTC» 500 6 Matasw) 

ESMath Set Ed. Board (TC=5oo 3 me.u ppi 
E5MD Assm. Consort. (TC»ooo 3per.vsT) 

E5Nat'l Coun. of Tchrs. of Math (TC3 50(; bMAT.VS.Av \ 
E5Nat'l Ctr. for Res. on Eval. iTC=i50 6.\sswHa w 
E5Pandey. Tej (TC«5oo3Sa.mm.aa) 

ESPfeififer (TCs5oo 3MmgaP) 

E5Reg. Ed Lab Net Prog (TC=ooo6T(X)Kip:) 

E6Bagley iTCs 500 6.assstd> 

E6Bellingham Pub Sch., (Tc:»ooo 3 belpi si 
EbCrowley (TC»50o.3sti’map) 

EbiCnight (TC»530 3HOWR S) 

E6Koretz (Tc« 000 sverpof) 

E6Kulm )TC» 5oo smatasw) 

E6Lambdjn (TCksoo splapoa) 

E6Nat'l Coun of Tchrs. of Math (TC» 500 6 M \t asav) 
E6Nat'l Ctr. for Res. on Eval. (TC=i50 6.vsswHav) 
E60R New Stand. Proj 'Tc» 000 3STn>OQ) 

E6Pandey (TC»5oo 3 Samm.a.a) 

EbPaulson (TC»50o sporglt) 

EbPritchard (TC»5oo sstitot) 

E6Reg. Ed. Lab Net. Prog. (TC>-ooo 6 T(X)kiP 2 ) 
EbSerrano (TC» 000 jloopoa) 

EbThompson (TC»5oo 3PormoD) 

E6VA DOE (TC»500 3GRAFOB) 

E6VA DOE (TC»500 3RESB(X3) 

E6V'A DOE (TC»500 3TEAGL I) 

E6VT DOE (TC»500 3GRAEIB) 

EbWilkins (TC» 530 3STI?0A) 

E7Aurora Pub Schools (Tc»ooo 3 scim,vP) 

E7Bums nca5oo 5 m.atlit) 

E7Burton (Tc«5oo 5 addsek) 

E7CT DOE (TC»000 3CONCOC) 

E'/Doig (TC« 500 3.ACTASM) 

ETEd. Testing Service (TCk 000 3MISai,.V) 

E7Finston (Tc»5oo 3STt.Asn 
E7Hom (TCKOOO 3LANC0P) 

E7Katims fTC» 500 spacket) 

E7Kulm (T."* 50 C CMATASW) 

E7MD Assm. Consort. (Tc«ooo3PERas1) 

E7Nicoll (TC«500.3MTDIAC2) 

E'^Pfeiffer, Sherroit (Tc«5oo 3NIMcap) 

ETReg, Ed Lab Net Prog. (TCkooo 6 T(X)MP 2 > 
E7Webb (TC«5oo 6 coi.grv) 
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ESBuxton (TC»?oo s.addseK) 

ESDoig (TC» 500 3ACTASM) 

ESHall (TC»500 3ALBORN) 

E8Harve> (Tc»500 omatteo 
ESK aiims (Tc» soo 3PACkE-n 
ESKulm (TC» 500 6MAT.ASW> 

E8MD Assm Conson iTC=ooo .ipkr vsTi 
E8Nai'l Ctr for Res onE\al tl -I'D b vss\vnu'. 
ESPfeiffer iTC=5oo immoaPi 
E8Reg Ed Lab Net Prog iTC='M)i)bTo<ikir:i 
ESSecada iTC:> ‘oo 41 ntc-or! 

ESSempIc iTt=wjo 3PtRASSi 

E9CT DOE rrt«)i)(i 3 co\c(k i 

EloEd Testing Service rrc=‘i) 0 h\ vKpm vRi 

FI Alberta Ed ( rc»5oo idiaaiaP) 

FlAiberta Ed. (TC=5 oo .tDiPEXPi 

FI Aurora Pub Schools (TC»ooo.tsciM.tPi 

FIAZ Stud Assm. Prog tTC» 500 3aristm2i 

FI Badger ac»5oo 3 onthomi 

FI Bagiev iTC" 5oo 6ASSSTr>i 

FlBavter iTC= 5 oo6Matpem 

FIC.A .Assm Prog nc’= 5 oo vsamm \.\:i 

FICA DOE iTC»i)0O6sn STSi 

FICA DOE (TC»50o 3 Agi F.soi 

F1Co11iS(TC»50(I 3COI.ROM1 

F ICT DOE (TC»ooo 3CONCfX' 

FICTT McGraw-Kill (TC»5oo 3 cthmai i 
F lCurriculum Corp (Tc»5oo . vm.aici Pi 
F IDoig (TC» 500 3.ACT ASM) 

F'.Ed Testing Service (Tc»5oo6\ae;pmari 
FI Ed Testing Service iTC»ooo amis u.ai 
F lFinston (Tc»5oo Asn ASt 1 
FI Hartman (TC»5oo am.atpoo) 

FI Hibbard (TCaOOO 6T(XiSTCi 
FlHibbard )TC=5oo aseli sp) 

FlKjUght (TC»530 AHOWtl SI 
F IKuim (TC» 500 6MAT.\SW I 
F ILane (Tc<«5oo aquacoa) 

FlLarter fTC»too6BENCHM) 

F 1 Lawrence (Tc*ooo.6INT',t:' : 

F 1 Lawrence (Tc»500.3tTACoc) 

FI Lehman ac*500.3pERASMi 
F IMadaus (Tc» 000 sbriexai 
F lMath Sci Ed Board (TC»5oo amf.utp) 
FIMD Assm Conson (TC» 0 (Ao amarasci 
FIMD Assm Conson (TC»ooo appr.vsd 
FIMD DOE (TC* 500 amspapmi 
F iMeluter (Tc»oio asihofpi 
F 'M odels of Auth Assm iTC»soo asiptf M) 
FINC Dept of Pub. Instr (Tc» 5oo ai iNct n 
F 1 Ncwmann (Tc» 050 30i i At 11 



FlNiCOll (TC»500 3MTD1AC2) 

FI OR DOE (TC»500 aordips) 

FIOR New Stand. Proj <YC>ooo3STt pogi 
FlPaulSOn iTC »500 6POROtT) 

F 1 Perlman I Tc» 000 acpspea) 

FlPsvch Corp (TC»500 AIVT.ASM) 

FlPsvch Corp iTCaAio aixa.ai.S) 

FlQin iTCs 500 AMNM.ATO) 

FlReg Ed Lab Net Prog iTC= 000 orookip:! 
F 1 Semple I TC *600 aper.vss i 
F lShoecraft iTc«5ooam(avitmi 
FlThompson (TC*5oo apor.modi 
FI Webb (TC»<oo6Coic)Rvi 
F lWhetton (TC«')‘’o AKEt wai 

F2Alberta Ed. (TC»5 oo adi.am.api 
F2Albena Ed (TC»5oo adipe.xpi 
F2Aurora Pub Schools (TC»ooo ascim.api 
F2AZ Stud. Assm Prog.dc* 500 a.arjstm2) 
F2Bagley (Tc» 500 6,assstdi 
F2CA Assm Prog. (TC»5oo asammaaI) 

F2CA DOE (TC* 0006 STL STSI 
F2CA DOE (Tc»500 3.Agi ESO) 

F2CT DOE (TC»ooo aconc(ac) 

F2CTB McGraw-Hill (TC»5oo .actbm.aT) 
F2Cumculum Corp (TC»5oo amatci Pi 
F2Doig (TC* 500 AACT.ASMl 
K2Ed Testing Service dc* 000 amisa.ai 
F2Finston (TC»5oo astt asit 
F2Hartman itc»5oo amatpooi 
F 2 Hibbard (TC»ooo 6T(xastc) 

F2Hibbard iTC»5oo ASEi.t sPi 
F2Kulm (TC* 500 6MAT.ASW1 
F2Lawrence (Tc*ooo 6 INTITca i 
F2Lawrence (TC*5oo ait.ac(X) 

F2Madaus iTC* 000 6 BR 1 EX^\i 

F2Math Science Ed Board iTc*5oo ame u pp) 

F2MD Assm Conson (TC*ooo3Mar.\so 

F2MD Assm Conson. (TC* 000 apep./\st) 

F2Models of Auth. Assm. (Tc»500 asittemi 

F2Newmann (TC* o5o AOHAni 

F20R New Stand Proj. (TC* 000 asia ■ptx?) 

F2Perlman (TC» 000 acpspeai 

F2Reg Ed Lab Net Prog. (TC»ooo 6 Tcxakip 2 ) 

F2Riverside Pub Co (TC* 5 oo acai.pru 

FtMD Assm Conson (TC»oooai’ER/\sti 

FtPerlman (TC* 000 acpspeai 

FA Reg Ed Lab Net Prog (Tc» 000 6TfxiWP2) 

F4Alberta Ed ( rc*5oo adiamap) 

F4Alper (Tc» 5oo av7J>imp) 

F4Aurora Pub SchooL (Tc*ooo ascimapi 
F 4AZ Stud. Assm. P'.og ac* 500 aarjstmai 
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F4Badger (Tc»50o 30 sthomi 
F 4CA Assm. Prog. (TC»$oo 3 sa.mm.a.a:i 
F4CA DOE (TC# 000 6STLSTS) 

F4Charles fTcs«500 6 HOWToE) 

F4Crowley <Tc»500 3S'n.'MAP) 

F4CT DOE(TC»ooo 3CONCOC) 

F4CTB f.lcGraw-Hill iTC=V)o 3cthm aTp 
F4Cumculum Corp 3 M aki Pi 

F4Doig(TC» 500 3ACTASMI 
F4Ed. Testing Ser\ice iTc=?oo t>\ u pmaki 
F 4Ft Hays Ed Dev Center (TC= Sio ^ST VVSM) 
F4Grccnwood (TC»?oo tow.wV} 

F4HaJI tTC»500 3A1 BORN) 

F4Hart man < rc »5oo 3 m a tpo< > » 

F4Hom (TCsooo n vncoP) 

F4Hyncs iTc»50u 3K.?\i,AP< 

F4Kalims iTC» 500 3packet> 

F4KuJm (Tc» ?ooe>M AT.ASW^ 

F4Lanc iTC»5oo acoa) 

F4Lartcr (TC»ioo 6BEscHsn 
F4Lawrcncc (Tc»ooo6INtttcv) 

F4Lawrcncc (Tc»5oo 31 tac(K) 

F4Lcach <Tc»5oo 3 ALTFoe) 

F4Lchman (Tc»5oo 3 PER,vsM) 

F4Math Science Ed Board uii \i pp 

F4MD Assm Consort uoo 3 m ak.vsc ) 
F4MD Assm Consort dc^^oon 3 pp.r,\5T) 
F4MD DOE (TC» 500 3Msp\pm) 

F4NC Dtpi of Pub Insir rre^ ^00 3i i\ci n 
F4Neumann (Tc»o5o 3(3iiAn^ 

F4NiC0ll (TC»500 3MTDIAC2) 

F40R DOE (TC3500 30RDIPS) 

F4UR New Stand. Proj. (Tc»ooo 
F4Paulson (Tc»5oo sporgip) 

F4Perlman <tc»ooo 3CPspe A) 

F4Pfeiffer dc»5oo 3 MMg.ap^ 

F4Ps>ch Corp (TCa 500 3.a.m.ap.M) 

F4Ps>ch Corp (TC»500 3PST.ASM^ 

F4Ps>ch Corp (TC»510 3GO.AL5) 

F4Reg, Ed Lab. Net. Prog, dc# uoo 6 Toomp 2 i 
F 4Rjverside Pub. Co. (Tc«5oo 3 calprd 
F 4Thompson rrc#5oo.3K)RMOD) 

F4VA DOE (TC«500 30RAEUB) 

F4VA DOE (TC#500 3TEAOn) 

F4 VT DOE dC«500 30RAErB) 

F4Webb dc»500 s^vltstf) 

F5 Aurora Public Schools dc»ooo >sc im api 
F 5CT DOE (Tc*MX)o 3CONCoc ) 

FSFinslon dc»5oo 3sn asd 
FSKatims dc« 5oo 3 PackeT) 

F5Kulm dC» 5f)0 6 MAT.A 5 ;W) 

F5Lcach (TC*5 '^k) 3ALTFoE) 

F5Lehman dc»5oo 3 PERasm) 



F5Pcrlman dc» 000 3cpspea) 

F5Reg. Ed. Lab Net Prog dc»ooo 6 TOOKip 2 > 

F6AZ Stud. Assm Prog dc» 500 3,arjs r\i2) 

F6Baglev dc» 50O6 .assstdi 

F6CA Assm. Prog dc»50o 3S,amma.a:) 

F6CA DOE dc»5oo 3,AQrESO) 

F6Crowley dc»50o 3STt. m.aP) 

F6CT DOEdc»ooo 3cONC(x:) 

F6Ed Testing SerMCC dc» 000 3mis.ai A) 
F6Finston dc»5oo 3 STV.asd 
F6Ft. Hays Ed. Dev Center dc» 500 3st,a.a.smi 
F 6HaJl dC»500 3,ALBGRN) 

F6Hartman dc»5oo 3 Matpoo) 

F6Hibbard dc»ooo 6 T(X}STo 
F6Hibbard dc»5oo 3SEU SP) 

F6Hom (Tc«ooo 3 Lancop) 

F6Kulm dc» 500 smatasw) 

F6Lambdin dc«500 3 PLak).a) 

F6Lane dc»500 3Qi acoa) 

F6Lchman dc»500 3 PERasm) 

FCiVlath. Sci. Ed. Board dc»500 3ME.Ai'?P) 
F6MD Assm Consort. dc» 000 3 PER.asT) 
F6MD DOE dC« 500 3MSP.APM) 

F6Mead (Tc»5oo 3IA£PP.A) 

F6Newmann dc» o5o 3GnAri) 

F6NiC0ll dC»500 3MTDIAC2) 

F60R DOE dc»5oo 30 RDIPS) 

F60R New Stand. Proj. dc» 000 3STVP<x^) 
F6Perlman dc» 000 3 cpspea) 

F6Psych. Corp (TC« 500 3AMAFAn 
F6Psych. Corp. dc»5oo 3 INT.asM) 

F6Psych. Corp. dc»5io 3 Goals) 

F6Reg. Ed Lab Net Prog, dc# ooo 6 Tookip 2 ) 
F6Riverside Pub Co dc»5oo tc.uyRi j 
F6Thompson dc»5oo jpormoo) 

F6VA DOE iTCff500 3 GR afoB) 

F6VA DOE (TC»500 dE.AGll) 

F6VT DOE dc»500 3 GR.AEIB) 

F^Baglcy dc» 5 oo6ASSSTD) 

FTLambdin dc»5oo 3 PLApoa) 

F7Rcg. Ed. Lab. Net. Prog dc»ooo 6 T(X 3 KiP 2 ) 

F8CA Assm. Prog. (Tc«5oo 3 Sammaa2) 

F8CT DOE dC<KK)0 3CONCOC) 

F8CujTiculum Corp dc«5oo 3 MA’^ciT) 

F8Ft. Hays Ed. Dev Center dc« 500 3st,aasM) 
F8Hartman dc«5oo 3 matpoo> 

F8Hv'nCS (TC»500 3K-5MAP) 

F8Katims dc« 500 3Packet> 

FHKulm dc*» 5oo smatasW) 

F8Lane dc»5oo 3qiacoa) 

FSLarter (Tc«ioooBENcHM) 
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FSLehman (TC»5oo jperasm) 

F8MD Assm. Consort. (Tc»ooo 3perasti 
FSM ead (TC»500 31AEPPA) 

FSMclUcr (TCoOIO.SSLTlOFP) 

FXOR New Stand. Proj. (TC» ooo 3sn p<x;) 
FSPcrlmr \ (TC> ooo 3 CPspe.m 
F 8Ps>ch Corp ,TC»»oo MVTASM, 

F8Reg Ed Lab Net Prog i rt-- uoo r.nx)kip:i 
FSSemplc iTc=mm) »pi,rassi 

F9Ft Hays Ed Dcn Center iTC> 5oo 3ST.k-vsMi 
F9Reg Ed Lab Net Prog (Tc> ooo 6r(x)kip:i 

FloCenter for Talent Dev itc>::o 3yi eki Mi 
FloCrowley (TC=?oo 3stim.\P) 

FloCumculum Corp (Tc»v)o 3M.vrci Pt 
FloKloosterman «TC»?oo 3 ME.\bkmi 
F lOKnight (TC»?3o 3HOWM s) 
Flt)Larter(Tc»io(joBENCHM) 

FlOTittle (TC» 300 tvt.vT.vsy) 

G I Alberta Ed iTC»5oo 3 DI.a.\,i.\P) 

G I Alberta Ed iTC=500 3DIPE\Pi 
G 1 .Appalachia Ed Lab (TC» f>oo 3 ai.t viM i 
G1 Aurora Pub Schools (Tc»ooo iscim.u’i 
G1 Bagiev (TC> ?oo 6.assstd> 

G 1 Baxter (Tc® «oo 6 M.\tpe.ai 
GICA DOE iTC»300 3 .\yi Esoi 
GlClark (TC»?oo 3\iMmMi 
G1 Co11iS(TC=500 ,3C0I ROM! 

GICTB McGraw-Hill iTc»?oo icthmah 
G 1 Doig iTC» 500 3.ACT.ASM ) 

GlEd. Testing Service (TC»>oo 6\ >i:pMARi 
GlEd. Testing Service (TC»ooo 3 MIS.u a> 

G1 Hibbard iTCsooos nxjSTO 
GlHibbard (TC»5 oo3SEUsp> 

GlKulm (TO 500 6M.AT.ASW') 

GIKY DOE (TC>060 3KIRIS94) 

GlLane (TC»50 o 3qlacoa) 

G 1 Lawrence (Tc»ooo ersTiTCv) 

G1 Lawrence (TC»5oo jltacoo 
G 1 MadauS (TC« OOO 6BR1EXA) 

GlMath Sci. Ed. Board (T c»5oo3ME.ai pp> 
GIMD Assm. Consort. (Tc» ooo 3 Marasc> 
GIMD Assm. Consort. frc» ooo 3 PERast> 
GIMD TX5E (TC» 50O3Mspapm) 

GIMD DOE (TC»ooo 3 Comst> 

G 1 Mead (Tc»5oo 3Iaeppa) 

GlMcltzer (Tc»oio 3 SIROfp> 

G1 Models of Auth Assm. (TC»5oo 3SL PTFM) 

G 1 Pfeiffer (Tc»?oo 3Nimoap> 

G1 Psych Corp (Tc»5io3tX)Al.S) 

GIReg Ed Lab Net Prog (TC»(HX)6r(x>MP:) 
G 1 Riverside Pi’h Co (TC»5oo 3 cai pri > 



GlSemple (Tc»600 3perass> 

G 1 Webb ( rc»5oo scolgrv) 

GlWhettOn (TC»070 3kEY193> 

G2Ed. Testing Service >Tc= ooo .mis ai.ai 
G2Kulm (Tc» 500 6 M.atasw > 

G2Perlman (Tc» ooo 3 CPSPEa> 

G2Psvch Corp (Tc> 5oo 3 a.map m> 

G2Reg Ed Lab Net. Prog. iTC'>(joo5T(H)kip:i 

G2.AZ Stud Assm Prog.(TC> <oo 3\rjstm:i 
G3Bellingham Pub. Sch.. (Tc»ooo3bELPt s> 
G.1CA Assm. Prog. (Tc»500 3 Sammaa2> 
G3Charles (Tc»500 6 HOWToE) 

G3CTB McGraw-Hill (TC»060 3Cat-5»> 

G3Ed Testing Service (TC» ooo 3Mis.Ai.At 
G3Hynes (Tc»500 3K-5 M,vP) 

G3Kulm (Tc» 500 6Matasw> 

G3MD Assm. Consort (TC»ooo 3Marasc> 
G3MD DOE (TC«ooo 3CoMST) 

G3Meltzer (TC»oio 3 SLTIofP) 

G3NC Dept, of Pub. Instr. (Tc» soo 3U\cn> 
G3Pandey (TC»50o 3sammaa> 

G3Paulson (TC»500 6porolT) 

G.tPerlm'n (TC»ooo 3CPspea' 

GiPsych Corp (TC» soo 3AM/U>-M) 

G3Psych. Corp (TC»soo 3 INT.asm> 

G.3Reg. Ed. Lab Net. Prog. (TC»ooo6Ttx)kiP2> 
G3Rjverside Pub Co (TC»5oo 3C^*a.PRi.> 
G3Thompson (Tc»500 3Pormod' 

G4Charles (TC»500 showtoe > 

G4Clamdge (Tc» i5o 6 Impnee> 

G4Curriciilum Corp. (TC»500 3Matct p> 

G4Ed. Testing Service (TC» ooo 3M1 s.ai.a> 

G4Ft Hays Ed. Dev. Center (TC» 5oo 3 st aasm> 
G4Hall (TC*500 3ALBORN) 

G4Hartman(TC»50O 3MATP(X3) 

G4Hibbard (Tc»ooo 6 Tcx 3 STC) 

G4Hibbard (Tc»5oo 3sellsP) 

G4Hom rrc«ooo 3 La.ncoP) 

G4Kulm (TC» 500 6MATA.SW) 

G4Lambdin nc»5oo 3 PLapoa> 

G4Leach rrc»500 3ALTFOE) 

G4Lehman (Tc«5oo sperasm) 

G4Newmann rrc» o5o joliait) 

G40R DOE rrc»5oo 30RDIPS) 

G40R New Stand. Proj rrc« ooo ssmxxj) 
G4Perlman (Tc»ooo 3 cpspea> 

G4Psych. Corp. (TC»5oo 3 INTasm> 

G4Reg. Ed. Lab Net. Prog (TC»ooo6TfxiKiR3> 
G4VA DOE (TC»500 3ORAfOB) 

G4 VA DOE (TC»500 STEAGID 
G4VT DOE (TC»500 30RA£IB> 
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